This study was conducted by staff from Abt Associates, which administers CLEAR. Therefore, the review of this study was conducted by ICF Incorporated, which also administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.
Citation
Martinson, K., Harvill, E., & Schwartz, D. (2020). Implementation and Impact of a Goal-Oriented Coaching Program for Cash Assistance Recipients in Michigan: The Job Search Assistance Strategies Evaluation (Report No. 2020-73). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Highlights
- The study's objective was to examine the impact of Michigan Goal Progress Success (MI-GPS) and Application Eligibility Period/Partnership. Accountability. Training. Hope. (AEP/PATH) programs on employment, earnings, and public benefits receipt outcomes.
- The study used a randomized controlled trial to evaluate the difference in outcomes between participants in the MI-GPS and AEP/PATH programs. Using data from the National Directory of New Hires, administrative program data, and follow-up survey data, the authors conducted statistical models to compare differences in outcomes between applicants assigned to each of the two programs.
- The study found a statistically significant difference in the amount of SNAP benefits received, with MI-GPS participants receiving less than AEP/PATH participants.
- This study receives a high evidence rating. This means we are confident that the estimated effects are attributable to the MI-GPS and AEP/PATH programs, and not to other factors.
Intervention Examined
MI-GPS and AEP/PATH Programs
Features of the Intervention
The Temporary Assistance for Needy Families (TANF) program helps low-income adults become more self-sufficient by providing cash assistance and employment services. To receive benefits, the TANF program requires participation in a specified set of work activities, including job search assistance (JSA). Michigan's Application Eligibility Period/Partnership. Accountability. Training. Hope. (AEP/PATH) program was the original initiative, while Goal Progress Success (MI-GPS) was the redesigned program. A comparison of the original and enhanced programs was conducted to evaluate the effectiveness of different JSA strategies and identify which aspects have the most significant impact on increasing employment and sustaining a living wage for applicants.
The AEP/PATH and MI-GPS programs included orientation, services during the application period, and access to workshops, job developers, supportive services, and retention services (e.g., cash incentives for 30 days of continuous employment). The MI-GPS program focused on collaboration with coaches and goal-oriented coaching with new strengths-based tools and resources. TANF cash recipients in MI-GPS could pursue their goals through activities that did not count toward the federal work participation rate (WPR). In contrast, the AEP/PATH program focused on program compliance during orientation and work activities had to count toward the WPR. Services for both programs were provided through five Michigan Works! Offices. Both Michigan programs served 1- and 2- parent TANF cash assistance applicants and recipients who were required to work. The programs were jointly operated by the Michigan Department of Health and Human Services and Michigan Talent Investment Agency, the state's workforce agency.
Features of the Study
The study used a randomized controlled trial to evaluate the impact of two different intervention programs. A total of 2,081 eligible applicants attended orientation, consented to the study, and completed a baseline information form. Of the eligible applicants, 1,044 applicants were assigned to the AEP/PATH program group and 1,037 were assigned to the MI-GPS program group. Almost all (92%) study participants were female with an average age of 38, 60% were Black, 33% were White, 76% were never married, 36% had received a high school diploma or equivalent, and 49% were not working at the time of the assessment but had worked in the past 12 months.
The study was conducted between 2016 and 2018. Data sources included baseline intake forms, the National Directory of New Hires (NDNH) data, administrative program data, and a follow-up survey. The baseline intake forms were completed by study participants prior to random assignment. Follow-up NDNH data were obtained for 15 months (five quarters), TANF and SNAP data were obtained for nine months, and the follow-up survey was administered six months after random assignment. The authors used statistical models to compare differences in outcomes between applicants assigned to each of the two intervention programs.
Study Sites
The evaluation includes five Michigan Works! offices:
- Genesee Shiawassee Thumb Michigan Works!
- Highland Park Michigan Works! One-Stop Service Center
- Livonia Michigan Works! One-Stop Service Center
- Southgate Michigan Works! One-Stop Service Center - Downriver Community Conference
- Wayne Michigan Works! One-Stop Service Center
Findings
Employment
- The study found no statistically significant differences in employment between AEP/PATH and MI-GPS participants at follow-up.
Earnings and wages
- The study found no statistically significant differences in earnings between AEP/PATH and MI-GPS participants at follow-up.
Public benefits receipt
- The study found no statistically significant differences in the proportion of AEP/PATH or MI-GPS participants who received TANF or SNAP benefits at follow-up.
- However, the study found that SNAP benefits were $158 less for participants in MI-GPS compared to AEP/PATH; this difference was statistically significant.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the MI-GPS and AEP/PATH programs, and not to other factors.