Skip to main content
UNITED STATES DEPARTMENT OF LABOR

New insights into the back on track model's effects on opportunity youth outcomes: Opportunity works final evaluation report (Anderson et al., 2019)

Review Guidelines

Causal Evidence Guidelines

Absence of conflict of interest.

Citation

Anderson, T., Braga, B., Derrick-Mills, T., Dodkowitz, A., Elizabeth Peters, H., Runes, C., Winker, M., Okoli, A., & Spievack, N. (2019). New insights into the back on track model's effects on opportunity youth outcomes: Opportunity works final evaluation report. Urban Institute. [Comparison across three sites]

Highlights

  • The study’s objective was to examine the impact of the Back on Track framework on earning a high school credential and being disconnected from both work and school (that is, neither working nor enrolled in school). The authors investigated similar research questions for three other contrasts, the profiles of which can be found here:
  • South King County/Seattle site
  • Hartford site
  • Philadelphia site
  • Using propensity score matching, the study compared outcomes for students who participated in the Opportunity Works program to those of youth from other “business-as-usual” programs in the same cities. The study tested for differences in outcome means between the treatment group and matched comparison group. The primary data sources were a baseline survey and a follow-up survey conducted 12 to 17 months later. 
  • The study suggested that Opportunity Works participants were significantly less likely to be disconnected from both work and school than comparison group members.  
  • The causal evidence rating for this study is low. This means we are not confident that the estimated effects are attributable to the Back on Track framework; other factors are likely to have contributed. 

Intervention Examined

Back on Track

Features of the Intervention

The Back on Track framework aims to improve outcomes for “opportunity youth,” young people ages 16 to 24 who are not in school or meaningfully employed. Back on Track provides a framework that Opportunity Works sites could adapt to focus their activities and service delivery on helping young people find education and employment success. 

The Back on Track framework has three program phases. The first phase is an enriched preparation program that recruits youth who have not completed high school and provides curriculum, support, and coaching to help them complete a high school credential and prepare for college and career. The second phase, postsecondary/career bridging, focuses on youth who have obtained a high school credential or are very close to doing so, and prepares them for college courses and career-oriented professional training. The final phase is first-year support through participants’ first year of college or career. In this study, sites were primarily focused on the second phase of the framework. 

Features of the Study

Using propensity score matching, the study compared outcomes for students who participated in the Opportunity Works program to those of youth from other “business-as-usual” programs in the same cities that also aimed to help underemployed, out-of-school youth of the same ages. The study took place at three sites in Hartford, CT; Philadelphia, PA; and South King County/Seattle, WA. Participants were generally ages 18-24. In Hartford, most (93%) already had a high school credential at baseline, while, in Philadelphia, almost none (4%) did. In South King County, about 60% had a high school credential. Most participants were people of color. 

The analytic sample was 188 treatment participants and 67 matched comparison group participants. The study uses propensity score matching to match treatment participants with individuals from the comparison group who were most like them, considering demographic characteristics, educational attainment at baseline, and more. After constructing the matched comparison group, the study tested for differences in outcome means between the two groups. 

Findings

Employment

  • The study suggested that the two groups differed in the percentage disconnected from work and school: 21% of intervention participants were disconnected from both work and school compared to a significantly higher 46% percent of comparison group participants.   

Education and skills gains

  • The study suggested there were no significant difference in the share of treatment and comparison group participants who completed a high school credential.  

Considerations for Interpreting the Findings

The treatment and matched comparison groups differed in terms of the share of participants who were female. This pre-existing difference between the groups, and not the Back on Track framework, could be one factor explaining the observed difference in outcomes between the groups.  

Causal Evidence Rating

The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar on all important characteristics before the intervention. This means we are not confident that the estimated effects are attributable to the Back on Track framework; other factors are likely to have contributed.  

Reviewed by CLEAR

January 2023

Topic Area