Absence of conflict of interest: If the study was conducted by Mathematica, we have an external consultant complete an independent review of the study. Please include the following language when that instance arises: “This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Meckstroth, A., Person, A., Moore, Q., Burwick, A., McGuirk, A., Ponza, M., Marsh, S., Novak, T., Zhao, Z., & Wheeler, J. (2008). Testing case management in a rural context: An impact analysis of the Illinois Future Steps program: Findings from the Rural Welfare-to-Work Strategies Demonstration evaluation. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Highlights
- The study’s objective was to examine the impact of the Future Steps rural welfare-to-work program on employment, earnings, and public benefit receipt.
- The authors randomly assigned Temporary Assistance for Needy Families (TANF) and Food Stamps recipients required to work to either a treatment group, which received job search and other services, or a control group, which did not. The authors evaluated the treatment’s impact using Illinois administrative data and follow-up surveys.
- The authors found that the Future Steps program increased the likelihood of being employed 30 months after enrollment by 8.6 percentage points.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Future Steps rural welfare-to-work program and not to other factors.
Intervention Examined
The Future Steps Rural Welfare-to-Work Program
Features of the Intervention
Treatment group members received employment-focused case management services, including (1) an assessment of skills and interests; (2) individualized job search, job placement, and skills enhancement programs; (3) referrals, supportive service payments (up to $500), and mentoring to overcome personal and logistical obstacles; and (4) extended postemployment support up to three months after securing a job. Some clients in the treatment group were volunteers, but others were required to participate or face a reduction in their TANF or Food Stamps benefits.
Features of the Study
Starting in 2001, the authors randomly assigned 630 low-income workers in rural, southern Illinois to the treatment or the control group in even proportions. Some sample members were TANF and Food Stamps recipients required to work to remain eligible for benefits; others were low-income volunteers willing and able to work 30 hours per week. Members of the treatment group participated in the Future Steps program, whereas members of the control group did not, but could continue to receive benefits and other services as usual.
The authors estimated the effect of the program by comparing the average outcomes from participant surveys and Illinois administrative data among a group of study participants randomly selected to participate in the program against those among a group randomly selected as a control, after adjusting for chance differences between the groups before the intervention.
Findings
- The authors found that the Future Steps program increased the likelihood of being employed 30 months after enrollment by 8.6 percentage points.
Considerations for Interpreting the Findings
This was the second in a series of reports on this evaluation of the Future Steps rural welfare-to-work program. Subsequent reports provided information on the program’s longer-term effects.
The study authors estimated multiple related impacts on outcomes related to employment. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
The authors noted the estimated impact might understate the effect of programs modeled on Future Steps because many treatment group members received few or insufficient services through the program and Future Steps did not capitalize effectively on the local community college’s resources.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Future Steps rural welfare-to-work program and not to other factors.