Citation
Hendra, R., Dillman, K-N., Hamilton, G., Lundquist, E., Martinson, K., Wavelet, M., Hill, A., & Williams, S. (2010). The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. [PROGRESS—Eugene]
Highlights
- The study’s objective was to estimate the impact of post-employment job supports on former welfare recipients’ employment and benefits receipt outcomes after three years. The Eugene site was one of four in Oregon that participated in the Employment Retention and Advancement (ERA) project.
- The authors randomly assigned 1,179 employed single parents who were former Temporary Assistance for Needy Families (TANF) program participants to either a treatment group that received employment retention services from Eugene’s Progress Towards Retention, Opportunities, Growth, Enhancement and Self-Sufficiency (PROGRESS) program or a control group that could participate in other employment-related programs in the community. The authors analyzed data from Unemployment Insurance (UI), Temporary Assistance for Needy Families (TANF), and Food Stamps administrative records and a survey administered 12 months after random assignment.
- The study found that those in the PROGRESS group were 7 percentage points less likely than control group members to be employed in the third year after random assignment and 6.6 percentage points less likely to have been employed in every quarter of that year.
- The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Eugene ERA program and not to other factors.
Intervention Examined
The Employment Retention and Advancement Project, Eugene
Features of the Intervention
The ERA project was introduced in 1999 as a nationwide exploration of factors that helped welfare recipients not only find employment but retain their positions and advance in their careers. Eugene, Oregon, was one of 16 sites across the United States, including 4 in Oregon, to receive funding from the U.S. Department of Health and Human Services to implement a program intended to improve welfare recipients’ employment outcomes.
Eugene’s ERA project was a voluntary post-employment program that provided retention and career-focused case management. A case manager from the Oregon Department of Human Services and a career development specialist from Lane Community College worked with each client in Oregon Department of Human Services branch offices. The team created a career development plan, including a needs assessment, career goals, and action steps, and documented any need for vocational assessments. Subsequent meeting topics depended on a participant’s needs but included job coaching, education and training referrals, and counseling.
Features of the Study
From June 2002 to June 2004, evaluators randomly assigned employed people who were leaving TANF, most of whom were single mothers and were underemployed. These clients had completed an “I Have a Job” form required of newly employed TANF applicants and TANF leavers working at least 20 hours per week. Evaluators randomly assigned clients who had completed this form without contacting them or confirming their interest in participating. Participants were enrolled from June 2002 through June 2004. The analysis included 1,179 participants, 585 in the treatment condition and 594 in the control condition. About 93 percent of participants were female, 85 percent were white non-Hispanic, and the average age was 29.4 years.
The authors estimated employment and earnings impacts by comparing regression-adjusted UI wage records of treatment and control group members over the three years after random assignment. The authors also calculated program impacts for benefits receipt measures, including whether participants received TANF or Food Stamps, through three years after random assignment using TANF and Food Stamps administrative records. Finally, the authors estimated impacts on self-reported employment and earnings using data from a 12-month survey administered to a subsample of study participants.
Findings
- The study found that those in the Eugene ERA group were 7 percentage points less likely than control group members to be employed in the third year after random assignment, significant at the 1 percent level.
- Treatment group members were also 6.6 percentage points less likely to have been employed in every quarter of the third year after random assignment, a difference that was statistically significant at the 5 percent level.
Considerations for Interpreting the Findings
The authors noted that many participants had already lost their jobs at the time of program entry. Baseline data showed that about 90 percent of participants were employed during the quarter of program entry. Because not all participants were employed, program staff reported having to spend large amounts of time on job-search assistance and reemployment instead of career counseling and job retention services.
The study authors estimated multiple related impacts on outcomes related to employment, earnings, and benefits receipt. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not report that they performed statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Eugene ERA project and not to other factors.