Absence of conflict of interest.
Citation
Highlights
-
The study's objective was to examine the impact of Scholars at Work (SAW) on college enrollment, wages, and employment.
-
Using a propensity score weighting approach, this study compares outcomes between students that participated in the SAW program to students that were randomly not invited to participate in New York City’s Summer Youth Employment Program (SYEP). Employment and earnings data were obtained through New York’s Department of Labor. College enrollment and student demographics data were obtained through New York’s Department of Education. The authors used student sex, race, poverty status, disability status, English language learner status, individualized education plan status, algebra state test scores, and graduation year to calculate propensity score weights. Since SAW had two components (a career exploration module and an internship module), the authors conduct their study on two separate samples, one for each module. Using these weights, the authors construct two separate control groups from students denied from the SYEP program: those enrolled in NYC schools offering Career and Technical Education (CTE) programs and those enrolled in NYC schools without CTE programs.
-
The authors find a positive association between both components of the SAW program and earnings in the first year post high school graduation. However, the authors do not find any relationship between SAW and postsecondary enrollment nor employment outcomes across any combination of the career exploration/internship treatment groups and the CTE/non-CTE control groups.
-
The quality of causal evidence presented in this report on education outcomes is moderate. Although randomization was compromised, the authors demonstrated baseline equivalence in after applying propensity score weighting. This means we are somewhat confident that the estimated effects are attributable to SAW, but other factors might also have contributed. However, the quality of causal evidence for employment and earnings outcomes is low, since the authors did not account for all other factors that could have affected the difference between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to SAW; other factors are likely to have contributed.
Intervention Examined
Scholars at Work (SAW)
Features of the Intervention
In 2009, Scholars at Work (SAW) was created through the partnership between New York City Department of Small Business Services and the Department of Education to develop skills needed for the workplace for high school students. Through SAW, high school seniors enrolled in Career and Technical Education programs could opt into a career exploration course or an internship. In the career exploration module, students learned about career opportunities and workplace skills. For the paid internships, students worked with a local business for a 13-week period after school. The treatment group consists of 233 students that participated in the career exploration module, and 388 students participated in the internship portion of SAW spanning from academic years 2010-11 through 2015-16. The comparison group contains 16,377 SYEP denied applicants who were attending schools that offered CTE programs, and 6,115 SYEP denied applicants who were attending schools that did not offer CTE programs.
Features of the Study
This study utilizes a propensity score weighting (PSW) approach and compares labor-related outcomes between students that participated in SAW to students that were randomly not selected into NYC's Summer Youth Employment Program (SYEP). The PSW algorithm adjusts the weighting of pretreatment student sex, race, poverty, disability, ELL, IEP, algebra score, graduation year, and advanced diploma recipient to better balance the comparison group to the treatment group. The authors compare the outcomes between the treatment group and the reweighted comparison group to study the impact that SAW had on college enrollment, employment, and earnings. Since students opted into the SAW program, the authors rely on the lottery system of SYEP to create a control group. The treatment group consists of 233 students that participated in the career exploration module, and 388 students participated in the internship portion of SAW spanning from academic years 2010-11 through 2015-16. The comparison group contains 16,377 SYEP denied applicants who were attending schools that offered CTE programs, and 6,115 SYEP denied applicants who were attending schools that did not offer CTE programs. This study relied on data from several sources. Data from the NYC Department of Youth and Community Development was used to identify the SYEP denied applicants from 2001-2016; NYC DOE provided student demographic data; SBS provided SAW enrollment and administrative data spanning from academic years 2010-11 through 2016-17.
Study Sites
The participating sites for SAW include all NYC schools offering Career and Technical Education programs.
Findings
Employment
- The study found no relationship between SAW participation and employment in the year following high school graduation.
Earnings
- The study found a positive relationship between SAW participation and earnings in the year following high school graduation.
Education
- The study found no relationship between SAW participation and college enrollment in the year following high school graduation.
Considerations for Interpreting the Findings
The authors did not account for all other factors that could have affected the difference between the treatment and comparison groups when examining employment and wages. In addition, pre-program employment and wages are not accounted for within the analysis, which may impact the findings for these outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report for findings examining education is moderate because randomization was compromised, though the authors demonstrated baseline equivalence after propensity score weighting was applied. This means we are somewhat confident that the estimated effects are attributable to SAW, but other factors might also have contributed.
The quality of causal evidence presented in this report for findings examining employment and earnings is low because the authors did not account for the factors that might have affected the differences between treatment and comparison groups when examining these outcomes. This means we are not confident that the estimated effects are attributable to SAW; other factors are likely to have contributed.