Citation
Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W. E. Upjohn Institute for Employment Research. [Apprenticeships]
Highlights
- The study’s objective was to examine the impact of the Workforce Investment Act apprenticeship programs in Washington State on participants’ employment rates, earnings, and public benefit receipt.
- The authors used a nonexperimental design to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefit receipt between those who took part in the apprenticeship programs and those who registered for employment services at the state Labor Exchange.
- The study found that, compared with those who registered for services at the Labor Exchange, participants in the apprentice programs had higher employment rates, more hours worked and greater quarterly earnings. Receipt of Unemployment Insurance benefits was higher for those in the apprenticeship group, and receipt of other public assistance was lower.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the apprenticeship programs; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
Workforce Investment Act (WIA) Apprenticeship Programs
Features of the Intervention
Apprenticeship programs were workforce development programs administered by the Washington State Department of Labor and Industries. The apprenticeships usually lasted multiple years and included on-the-job training and classroom instruction. The programs typically included a minimum of 2,000 hours of work and 144 hours of formal instruction.
Features of the Study
The authors used a nonexperimental statistical procedure (propensity-score matching) to create a comparison group of people who registered for services at the Labor Exchange and were similar to apprenticeship program participants in terms of gender, race, ethnicity, age, location, and employment history. The authors collected Unemployment Insurance records for those who had exited an apprenticeship program or Labor Exchange from July 2001 to June 2002 to estimate the long-term impacts of the program in quarters 9 to 12 after program exit. The authors also collected Unemployment Insurance records for those who exited an apprenticeship program or Labor Exchange from July 2003 to June 2004 to estimate the short-term impacts in the 3 quarters following program exit. The 2,895 apprenticeship program participants who exited in 2001–2002 were matched to a sample of 2,701 comparison group members, which was drawn from the 188,282 adults who registered at the Labor Exchange. The 2,409 apprenticeship program participants who exited in 2003–2004 were matched to a sample of 2,222 comparison group members, which was drawn from the 164,811 adults who registered at the Labor Exchange. The authors then compared the employment, hourly wages, hours worked per quarter, quarterly earnings, and receipt of benefits of the apprenticeship and comparison groups before and after participation.
Findings
- Employment. The authors reported that the percentage of quarters employed for those who took part in an apprenticeship program increased by 7.4 percentage points 3 quarters after program exit compared with the comparison group. The percentage of quarters employed for the apprenticeship group increased by 6.8 percentage points in quarters 9 to 12 after program exit compared with the comparison group. The average number of hours worked per quarter 3 quarters after program exit for apprenticeship program participants also increased by 33.5 hours and increased by 20.3 more 9 to 12 quarters after program exit relative to the comparison group.
- Earnings. The authors reported that, 3 quarters after program exit, average earnings per quarter increased by $2,730 and by $2,075 in quarters 9 to 12 after program exit for the apprenticeship program group compared with the comparison group. Average hourly wages increased by more for the apprenticeship group than the comparison group 3 quarters after program exit (by $6.60) and 9 to 12 quarters after program exit (by $5.08).
- Public benefit receipt. The authors found that, compared with those who registered for services at the Labor Exchange, the likelihood of receiving Unemployment Insurance benefits increased by 11.3 percentage points 3 quarters after program exit and by 14.1 percentage points in quarters 9 to 12 after program exit for those who took part in an apprenticeship program, with claimants receiving a significant average increase in benefits of $265 more 3 quarters after program exit and $194.40 more 9 to 12 quarters after exit relative to the comparison group. The rate and amount of Temporary Assistance for Needy Families benefits received decreased more for the apprenticeship group relative to the comparison group 9 to 12 quarters after program exit, whereas the rate and amount of food stamps received decreased more for the apprenticeship group 3 quarters after program exit and 9 to 12 quarters after exit. The rate of Medicaid receipt decreased both 3 and 9 to 12 quarters after exit for the apprenticeship group compared with the Labor Exchange group.
Considerations for Interpreting the Findings
Although the authors accounted for many characteristics of the apprenticeship and comparison groups in their analysis, the decision to define the groups based on their date of program exit rather than program entry is problematic. For example, if the average length of participation was 6 months for the apprenticeship group compared with one month for the Labor Exchange group and we compared the groups’ earnings 6 months after their recorded exit dates, we would see apprenticeship program participants’ earnings about 12 months after they started receiving services and Labor Exchange participants’ earnings about 7 months after they started receiving services. If everyone stayed on their original upward-sloping wage trajectory, it would appear as though the apprenticeship program participants earned more 6 months after their exit dates. However, this would not be attributable to receiving apprenticeship program services; it would be caused by the difference in elapsed time across the groups (12 months for apprenticeship program participants versus 7 months for Labor Exchange participants). Therefore, studies defining the groups based on exit date instead of entry date without accounting for program length can receive only a low evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the apprenticeship programs; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D.J. Besharov & P.H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings (pp. 347-370). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.