Absence of conflict of interest.
Citation
Shiferaw, L., Deutsch, J., Fischer, B., Geckeler, C., Paprocki, A., Folsom, L., Wiegand, A., Lewis, G., English, B. (2024). Employment programs to support reentry: Findings from the Reentry Project grant evaluation. Washington, DC: Chief Evaluation Office, U.S. Department of Labor. https://www.dol.gov/agencies/oasp/evaluation/completedstudies/Reentry-Projects-Grant-Evaluation
Highlights
- The study's objective was to examine the impact of the Reentry Project (RP) on employment and earnings.
- Using Workforce Integrated Performance System (WIPS), National Directory of New Hires (NDNH), and criminal justice data, the authors conducted a nonexperimental study to compare the outcomes of RP participants to a matched comparison group of Wagner-Peyser program participants.
- The study suggested that RP participants had significantly lower earnings and were significantly less likely to be employed in the 9th and 10th quarters after enrollment than the matched comparison group of Wagner-Peyser program participants.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Reentry Project; other factors are likely to have contributed.
Intervention Examined
Reentry Project (RP)
Features of the Intervention
The U.S. Department of Labor awarded RP grants between 2017 and 2019 to help organizations in communities with high crime implement reentry programming to improve the workforce and criminal justice outcomes of individuals who have been involved in the criminal justice system. RP programs targeted either adults aged 25 or older who were incarcerated in the adult criminal justice system and released from prison or jail within 180 days of enrollment or young adults aged 18 to 24 who had current or previous involvement in the juvenile or adult justice system.
RP programs offered career preparedness, employment-focused services, and case management. All RP programs were required to use at least one of the following strategies: registered apprenticeship, work-based learning, and career pathways. Across all grant recipients, RP programs were located in high-crime, high-poverty communities across the U.S.. Both community-based organizations and intermediary organizations implemented the program. The study is limited to six states in the U.S. (Alabama, Florida, New Jersey, New York, Oregon, and Pennsylvania) where the four largest RP intermediary grantees worked and where state agencies provided sufficient administrative data.
Features of the Study
The authors used a non-experimental design to compare the outcomes of RP participants to a matched comparison group of Wagner-Peyser participants. The study included individuals served by either RP or Wagner-Peyser programs between 2018 and 2021 in one of six states (Alabama, Florida, New Jersey, New York, Oregon, and Pennsylvania). The study sample included 1,198 RP participants and 16,032 Wagner-Peyser participants. The treatment group received RP grant-funded services. The comparison group received Wagner-Peyser employment services, including basic career services, individualized career services, and training or education. In the RP sample, 53% were in the adult program and 47% were in the young adult program. Additionally, 79% were men and 21% were women.
The authors used Workforce Integrated Performance System (WIPS), National Directory of New Hires (NDNH), and criminal justice data. The authors acquired WIPS data from program year 2018 through the second quarter of program year 2021 for all RP and Wagner-Peyser participants. Depending on the state, NDNH data included earnings and employment data starting between 2020 Q3 and 2021 Q4 and ending in 2023 Q3. The authors also used criminal justice data from the six states included in the study.
The authors used a matched comparison design to compare the outcomes of RP participants to Wagner-Peyser participants. The authors matched at the individual level using both exact matching and caliper matching based on propensity scores. The authors estimated impacts for RP participants with observable employment and earnings data for the confirmatory outcomes and for whom the authors found an acceptable matched comparison.
Study Sites
There were six sites in the study:
- Alabama
- Florida
- New Jersey
- New York
- Oregon
- Pennsylvania
Findings
- Earnings and wages. The study suggested that RP participants earned significantly less in the 9th and 10th quarters after enrollment than the matched comparison group of Wagner-Peyser participants.
- Employment. The study suggested that RP participants were significantly less likely to be employed in the 9th and 10th quarters after enrollment than the matched comparison group of Wagner-Peyser participants.
Considerations for Interpreting the Findings
The author did not account for preexisting differences between the groups before program participation or include sufficient control variables. These preexisting differences between the groups—and not RP—could explain the observed differences in outcomes. While the study captures a self-reported employment indicator upon program entry (from WIPS data) and uses this for exact matching with the comparison group, it does not provide insight into employment during the year before program enrollment. The lack of pre-program sentencing/incarceration data is also problematic because there could be unobserved discrepancies in whether Wagner-Peyser participants were incarcerated before the program, which could affect past and future employment and earnings. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Additionally, the authors only included 17% of all RP participants in the study due to limited data availability. Study participants were drawn from only six of the 34 states where RP operated, which raises questions about the generalizability of the findings to all RP participants nationwide. The study also included individuals who received services before and during the COVID-19 pandemic, so the pandemic may have influenced results in ways that are hard to separate from the program’s impacts. Finally, the analysis compares outcomes for RP participants and Wagner-Peyser participants, but does not examine differences between RP participants and individuals who received no employment services. Thus, the study limits understanding of the program’s impact compared to having no intervention at all.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Reentry Project; other factors are likely to have contributed.