Absence of conflict of interest.
Citation
Moore, Q., Wu, A., Kautz, T., Kent, C., McConnell, S., McInnis, N., Patnaik, A., & Schochet, O. (2024). Can a participant-centered approach to setting and pursuing goals help adults with low incomes become economically stable? Impacts of four employment coaching programs 21 months after enrollment. OPRE Report #2024-061. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. [Goal4 It!]
Highlights
- The study's objective was to evaluate the impact of four employment coaching programs serving Temporary Assistance for Needy Families (TANF) recipients and adults with low incomes on earnings and wages, employment, public benefits receipt, employer benefits receipt, and education and skill gains. This profile focuses on the Goal4 It! program. The authors investigated similar research questions at other programs; profiles of those studies are available here:
- MyGoals: Can a participant-centered approach to setting and pursuing goals help adults with low incomes become economically stable? Impacts of four employment coaching programs 21 months after enrollment (Moore et al., 2024) | CLEAR
- Family Development and Self-Sufficiency Program: Can a participant-centered approach to setting and pursuing goals help adults with low incomes become economically stable? Impacts of four employment coaching programs 21 months after enrollment (Moore et al., 2024) | CLEAR
- LIFT: Can a participant-centered approach to setting and pursuing goals help adults with low incomes become economically stable? Impacts of four employment coaching programs 21 months after enrollment (Moore et al., 2024) | CLEAR
- The Goal4 It! evaluation was a randomized controlled trial that assigned 802 participants to a treatment or a control group. Using participant surveys and administrative data, the authors used statistical models to compare the outcomes of treatment and control group participants.
- The study found that Goal4 It! participants self-reported being employed for significantly more time than control group participants at the nine-month follow-up.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to Goal4 It! and not to other factors.
Intervention Examined
Goal4 It!
Features of the Intervention
Goal4 It! is an employment coaching program that uses a structured, four-step approach in which participants set goals, develop plans, work to execute those plans, and then review progress and revise goals and plans as needed. The program was developed as an alternative to traditional TANF case management. The evaluation focused on a pilot implementation of Goal4 It! that was launched in Jefferson County, CO, in 2018.
Features of the Study
The study was a randomized controlled trial that assigned 802 TANF participants who were subject to work requirements to a treatment or a control group. Study participants were enrolled in the study from October 2018 to November 2019. The 401 study participants assigned to the treatment group were required to participate in Goal4 It! coaching. The 401 study participants assigned to the control group were required to participate in traditional TANF case management and could not access Goal4 It! coaching.
Most study participants were female (90 percent), and the average participant’s household included two children. Nearly half of study participants were White, non-Hispanic (47 percent), more than a two-fifths were Hispanic (42 percent), and 9 percent were Black, non-Hispanic. At baseline, nearly a third of study participants reported having worked for pay in the past 30 days (27 percent). About one in five participants did not have a high school diploma or GED (22 percent).
The study relied on multiple data sources, including a baseline survey at study enrollment, 9- and 21-month follow-up surveys, and administrative data on quarterly earnings, unemployment insurance, and public assistance receipt. The authors used statistical models to compare the outcomes of treatment and control group participants and survey weights to adjust for survey nonresponse.
Findings
- Earnings and wages. The study did not find statistically significant effects of Goal4It! on earnings.
- Employment. The study found that Goal4 It! group members self-reported being employed for significantly more time during the first nine months of the follow-up period (46 vs. 38 percent of months) than the control group.
- Employer benefits receipt. The study did not find statistically significant effects of Goal4 It! on employer benefits receipt.
- Public benefits receipt. The study did not find statistically significant effects of Goal4 It! on public benefits receipt.
- Education and skills gains. The study did not find statistically significant effects of Goal4 It! on education and skills gains.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to earnings and wages, employment, public benefits receipt, employer benefits receipt, and education and skills gains. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance, and not because they reflect program effectiveness. To help address this risk, the study authors pre-specified four confirmatory outcomes: a scaled measure of self-regulation and goal-related skills, earnings, a scaled measure of economic wellbeing, and TANF assistance receipt. However, the authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to Goal4 It! and not to other factors.