Citation
Hendra, R., Dillman, K.-N., Hamilton, G., Lundquist, E., Martinson, K., Wavelet, M., Hill, A., & Williams, S. (2010). The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. [VISION—Salem, OR]
Highlights
- The study’s objective was to estimate the impact of pre- and post-employment job supports and intensive case management on Temporary Assistance for Needy Families (TANF) recipients’ employment and benefits receipt outcomes after three years. The Salem site was one of several across the United States that participated in the Employment Retention and Advancement (ERA) project.
- The authors randomly assigned 1,504 single parents eligible for welfare-to-work programs to either a treatment group that received Valuing Individual Success and Increasing Opportunities Now (VISION) services or to a control group that received the state’s typical welfare-to-work program. The authors analyzed data from Unemployment Insurance (UI), TANF, and Food Stamps administrative records and a survey administered 12 months after random assignment.
- The study found that those in the VISION group were 6.7 percentage points more likely than those in the control group to have received TANF benefits at any point during the first follow-up year. The VISION group also received, on average, $298 more in annual TANF benefits during the first three follow-up years, compared with the control group.
- The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Salem ERA program and not to other factors.
Intervention Examined
The Salem Valuing Individual Success and Increasing Opportunities Now (VISION) Program
Features of the Intervention
The ERA project was introduced in 1999 as a nationwide exploration of factors that help welfare recipients not only find employment but retain their positions and advance in their careers. Salem was one of 16 sites across the United States to receive funding from the U.S. Department of Health and Human Services to implement a program intended to improve welfare recipients’ employment outcomes.
The local welfare agency and a community college ran the Salem VISION program and operated it out of a One-Stop Career Center. The program provided job search assistance and postemployment services to unemployed TANF recipients. Participation was typically mandatory and was enforced by sanctions. Job search assistance included job placement and workshops on job retention and career paths. Staff helped clients develop and regularly revisit a personal development plan to identify and prepare for a job that interested them. A case manager from the local welfare office could authorize supportive services. VISION staff reportedly devoted most of their time to pre-employment services and helped clients gain access to public assistance. VISION also offered post-employment services that focused on promoting job retention and career advancement, including support in achieving the career goals set before employment.
Features of the Study
From May 2002 to May 2004, 1,820 TANF applicants eligible for welfare-to-work programs were randomly assigned to VISION or the state’s typical welfare-to-work program, which was delivered at the local welfare office. This program also provided job search assistance but did not continue to provide services after participants found work. The analysis excluded the 316 participants from two-parent homes, and therefore analyzed 1,504 single parents.
The authors estimated employment and earnings impacts by comparing regression-adjusted UI wage records of treatment and control group members through three years after random assignment. The authors also calculated program impacts for benefits receipt measures, including whether participants received TANF or Food Stamps, through three years after random assignment using TANF and Food Stamps administrative records.
Findings
- The study found that those in the Salem ERA group were 6.7 percentage points more likely than members of the control group to have received TANF benefits at any point during the first follow-up year.
- Members of the treatment group also received, on average, $298 more in annual TANF benefits during the first three follow-up years compared with the control group. These differences were statistically significant.
Considerations for Interpreting the Findings
The authors mentioned that the Salem ERA program was successful in implementing pre-employment services, but struggled to provide post-employment services due to high staff turnover and a statewide hiring freeze.
Further, the authors estimated multiple related impacts on outcomes related to employment, earnings, and benefits receipt. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence provided in this study is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Salem ERA program and not to other factors.