Citation
Hendra, R., Dillman, K.-N., Hamilton, G., Lundquist, E., Martinson, K., Wavelet, M., Hill, A., & Williams, S. (2010). The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. [Training Focused and Work Plus—Riverside, California]
Highlights
- The study’s objective was to estimate the impact of post-employment job supports and intensive case management on welfare recipients’ employment and benefits receipt outcomes after four years. The Riverside Phase 2 site was one of two in Riverside, California, that participated in the Employment Retention and Advancement (ERA) project.
- The authors randomly assigned 3,029 employed single-parent participants in Riverside’s Temporary Assistance for Needy Families (TANF) program to one of two treatment groups (Work Plus and Training Focused) that received differing ERA services or to a control group that received standard TANF services. The authors analyzed data from Unemployment Insurance (UI), TANF, and Food Stamps administrative records and a survey administered 12 months after random assignment.
- The study found that, compared with the control group, those in the Riverside Work Plus group were 3.2 percentage points more likely to have received Food Stamps in the first year after random assignment.
- The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Riverside Work Plus program and not to other factors.
Intervention Examined
The Employment Retention and Advancement Project—Work Plus and Training Focused (Phase 2), Riverside
Features of the Intervention
The ERA project was introduced in 1999 as a nationwide exploration of factors that help welfare recipients not only find employment but retain their positions and advance in their careers. Riverside included 3 of 16 sites across the United States to receive funding from the U.S. Department of Health and Human Services to implement a program intended to improve welfare recipients’ employment outcomes. (The authors reported results for a third site in Riverside separately, so CLEAR also reviewed the results for that site separately.)
TANF recipients assigned to the Riverside Work Plus and the Training Focused programs could receive intensive case management and enhanced support services, all aimed at encouraging employment stability. In the Work Plus program, staff encouraged clients to attend education and training activities after working at least 20 hours per week. Participation in education and training activities was premised on the belief that working reinforces the value of education and training and that working while in school or in training increases the likelihood of program completion and the use of newly attained skills. In the Training Focused program, staff encouraged clients to attend education and training (including vocational training) activities, and clients could decrease or eliminate their work hours if they participated in education and training.
Both programs also provided supportive services (such as child care, transportation, and payments to participate in program activities); job search services; and referrals to social services (such as mental health, domestic violence, and substance abuse) and public assistance programs. Work Plus was implemented by the county welfare agency and Training Focused program was implemented by the county workforce development agency.
Features of the Study
From January 2001 to October 2003, 3,029 employed, single-parent TANF recipients were randomly assigned to one of two treatment groups, each of which received differing ERA services, or a control group, which received standard TANF services through the existing welfare-to-work “Work Focused” program. Clients in this program were eligible to receive TANF, Food Stamps, child care and transportation assistance, and medical coverage. They also received limited and reactive case management. The authors estimated employment and earnings impacts by comparing regression-adjusted UI wage records of treatment and control group members over the four years after random assignment.
The authors also calculated program impacts for benefits receipt measures, including whether participants received TANF or Food Stamps, through four years after random assignment using TANF and Food Stamps administrative records. Finally, the authors estimated additional employment and earnings outcomes from a survey administered 12 months after random assignment.
Findings
- The study found that, compared with the control group, those in the Riverside Work Plus group were 3.2 percentage points more likely to have received Food Stamps in the first year after random assignment. However, the study found no statistically significant effect on employment or earnings.
- The Riverside Training Focused program had no statistically significant effect on earnings, employment, or benefits receipt compared with the control group.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to employment, earnings, and benefits receipt. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not report that they performed statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Riverside Work Plus program and not to other factors. It also means that we are confident that any estimated effects would be attributable to Riverside Training Focused program and not to other factors. However, the study did not find statistically significant effects of the Training Focused program.