Citation
Hollenbeck, K. & Huang, W-J. (2014). Net impact and benefit-cost estimates of the workforce development system in Washington state. (Upjohn Institute Technical Report No. 13-029). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
Highlights
- The study’s objective was to examine the impact of participating in apprenticeship programs registered with the state of Washington on employment rates, earnings, and benefit receipt.
- The authors used a nonexperimental method to compare the short-term (3 quarters after program exit) and long-term (9–12 quarters after program exit) employment, earnings, and unemployment insurance benefits between those who took part in the apprenticeship programs and those who registered for services at the Labor Exchange (Employment Services).
- The study found that, compared to those who registered for services at the Labor Exchange, participants in the apprenticeship programs had higher employment rates, hours worked, earnings, and receipt of unemployment insurance benefits.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the apprenticeship programs; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
Registered Apprenticeship Programs
Features of the Intervention
Registered apprenticeship programs are workforce development programs administered by the Washington State Department of Labor and Industries. The apprenticeships usually last multiple years and include both on-the-job training and classroom instruction. In order to complete the programs, participants generally need to complete a minimum of 2,000 hours of work and 144 hours of formal instruction.
Features of the Study
To compare with the group in apprenticeship programs, the authors used a nonexperimental statistical approach called propensity score matching to create a similar comparison group of people who registered for services at the Labor Exchange. The authors then used a difference-in-difference model to compare the two groups on employment, hourly wages, hours worked per quarter, quarterly earnings, and quantity of benefits received before and after participation. The authors collected unemployment insurance records for those who had exited an apprenticeship program between July 2005 and June 2006 to estimate the long-term impacts of the program in quarters 9–12 after program exit; the authors also collected unemployment insurance records for those who exited between July 2007 and June 2008 to estimate the short-term impacts in the 3 quarters following program exit. There were 3,214 participants who exited in 2005–2006 and 4,082 who exited in 2007–2008.
Findings
Employment
- The authors reported that the percentage of quarters employed for those who took part in an apprenticeship program significantly increased by 7.8 percentage points more in the third quarter after program exit compared to those who registered for services at the Labor Exchange; the percentage of quarters employed for the apprenticeship group increased by 9.8 percentage points more in quarters 9–12 after program exit compared to the Labor Exchange group. The average number of hours worked per quarter also increased significantly more for those who took part in an apprenticeship program; the average hours worked per quarter by the treatment group increased by 53.7 more in the third quarter after program exit and increased by 50.9 more in quarters 9–12 after program exit than the comparison group.
Earnings and wages
- The authors reported that average earnings per quarter significantly increased by $3,243 more in the third quarter after program exit and by $3,511 more in quarters 9–12 after program exit for those who took part in an apprenticeship program compared to those who registered at the Labor Exchange. Average hourly wages significantly increased by more for the treatment group than the comparison group, both in the third quarter after program exit and in quarters 9–12 after program exit (by $6.03 and $7.27, respectively).
Public benefits receipt
- The authors found that, compared to those who registered for services at the Labor Exchange, the likelihood of benefit receipt significantly increased by 5.6 percentage points more in quarters 9–12 after program exit for those who took part in an apprenticeship program, with claimants receiving a significant average increase in wages of $74 more than the comparison group.
Considerations for Interpreting the Findings
Although the authors accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the authors’ decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the apprenticeship program and Labor Exchange participants were on identical wage trajectories before receiving services from their respective programs, and suppose that the average length of participation in the apprenticeship program was six months, whereas that for Labor Exchange was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would see apprenticeship program participants’ earnings about 12 months after they started receiving services and Labor Exchange participants’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the apprenticeship program participants earned more 6 months after their exit dates. However, this would not be attributable to receiving apprenticeship program services; it would be caused by the difference in elapsed time across the groups (12 months for apprenticeship program participants versus 7 months for Labor Exchange participants). Therefore, studies defining the groups based on exit date instead of entry date cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because treatment is confounded with systematic differences in outcome measurement follow-up length for treatment and comparison group members. This means we are not confident that the estimated effects are attributable to registered apprenticeship programs; other factors are likely to have contributed.