Citation
Dyke, A., Heinrich, C., Mueser, P., & Troske, K. (2005). The effects of welfare-to-work program activities on labor market outcomes. Institute for Research on Poverty, 1295(5). [North Carolina]
Highlights
- The study’s objective was to examine the impact of North Carolina’s welfare-to-work program training activities on the quarterly earnings of women ages 18 to 65 years in single-parent households who were cash recipients of Temporary Assistance for Needy Families (TANF).
- Using data from the state’s administrative records and unemployment insurance program, the authors compared the quarterly earnings of women who took part in the training activities with those who enrolled in the welfare-to-work program but never took part in any of the training.
- The study found that those taking part in any of the training activities experienced an initial (quarter 1) reduction in their quarterly earnings, compared with those who did not take part in the training activities. However, those who took part in the assessment or intensive training activities experienced an increase in both their cumulative (based on 16 quarters of follow-up) and average (based on quarters 11 to 15 of follow-up) quarterly earnings, compared with those who did not take part in any training activities.
- The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the welfare-to-work program, but other factors might also have contributed.
Intervention Examined
The Welfare-to-Work Program
Features of the Intervention
The North Carolina welfare-to-work program offered three different categories of training activities: (1) assessment, including formal paper-and-pencil testing and the development of a self-sufficiency plan; (2) job-search and job-readiness training; and (3) intensive training, which included work experience, vocational and technical skills training, postsecondary education courses, and basic education activities (public school attendance up to grade 12, English as a second language courses, literacy courses, and adult education courses that prepared the candidate for a high school equivalency diploma).
Most TANF recipients were required to participate in the training activities. The program exempted certain recipients from the program training activities, such as those with very young children, those facing too many obstacles to participate according to their caseworker, and those who already worked a minimum number of hours. Those required to participate in the program faced losing all benefits if they chose not to participate.
North Carolina’s welfare-to-work program adopted the work-first philosophy, meaning that it emphasized moving welfare recipients into jobs as quickly as possible. The program therefore primarily focused on job-search and job-readiness training.
Features of the Study
This nonexperimental analysis included 1,035,490 women ages 18 to 65 who became new cash recipients of TANF from the second quarter of 1997 to the fourth quarter of 1999 and were included in this nonexperimental analysis. The authors used a statistical model to compare the quarterly earnings of women who took part in any of the training activities of the welfare-to-work program and those who did not. The authors also compared those who participated in each category of training activity with those who chose not to participate at all. For example, the authors compared the quarterly earnings of participants who only completed the assessment with those who completed none of the training activities.
Findings
- The study found an initial (quarter 1) reduction in quarterly earnings of $332 for women who took part in any of the welfare-to-work program training activities, compared with those who took part in none of the training activities. This reduction was statistically significant (CLEAR calculations). The cumulative (based on 16 quarters of follow-up) and average (based on quarters 11 to 15 of follow-up) quarterly earnings of those who took part in any of the training activities also decreased by $343 and $9, respectively. The decrease in cumulative earnings was statistically significant, but the decrease in average quarterly earnings from quarters 11 to 15 was not (CLEAR calculations).
- Compared with those who took part in none of the training activities, those who took part in only the assessment activities experienced an initial reduction in quarterly earnings of $161, but an increase in both cumulative and average quarterly earnings of $801 and $20, respectively. The initial reduction and cumulative increase in earnings was statistically significant, but the increase in average quarterly earnings was not (CLEAR calculations).
- Compared with those who took part in none of the training activities, those who took part in the job-search and job-readiness activities experienced an initial reduction in quarterly earnings of $413 and decreased cumulative and average quarterly earnings of $884 and $82, respectively. All three findings were statistically significant (CLEAR calculations).
- Compared with those who took part in none of the training activities, those who took part in the intensive training activities experienced an initial reduction in quarterly earnings of $588, but an increase in both cumulative and average quarterly earnings of $534 and $215, respectively. All three findings were statistically significant (CLEAR calculations).
Considerations for Interpreting the Findings
The authors included each unique welfare enrollment spell individually in the analysis, meaning that individuals might be included in the study multiple times if they exited and reentered the TANF program. The analysis therefore estimated the impact of welfare-to-work program training activities for each entry into the TANF program. This approach could lead to impact estimates that appear to be more precise than they really are because it did not account for the identity of the individual experiencing multiple spells.
In addition, the authors noted that these results might be biased due to selection effects. For example, as participation in welfare-to-work program training activities was required for those who were unable to find employment, training might have occurred when these people were at a low level of earnings and therefore their earnings could be expected to improve regardless of the intervention. This would mean that the estimated impacts reflect the effects of such differences between the treatment and comparison groups as well as the impact of the welfare-to-work program training activities.
Finally, the study authors estimated multiple related impacts on outcomes related to quarterly earnings. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the welfare-to-work program training activities, but other factors might also have contributed.