Citation
Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [WIA Youth]
Highlights
- The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Youth program on the employment rate and earnings of low-income youth.
- The authors used a nonexperimental method to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefits receipt between those who took part in the WIA Youth program relative to those who registered for services at the Labor Exchange.
- The study found that, compared with those who registered for Labor Exchange services, WIA Youth program participants had lower employment, with mixed findings for earnings.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the WIA Youth program; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Workforce Investment Act (WIA) Youth Program
Features of the Intervention
The WIA Youth Program was authorized by Title I of the Workforce Investment Act of 1998 and was superseded by the Workforce Innovation and Opportunity Act (WIOA), effective in July 2015. The Youth Program services, which remained essentially the same under WIOA, were designed to provide quality employment and training services to eligible workers. Administered through local workforce investment areas, the Youth Program served people ages 14 to 21 years old through a set of core services; these included job placement assistance, skills assessments, and provision of information on the labor market. In addition, those unable to obtain a job through core services alone could receive intensive services—which included counseling and specialized assessments—and vouchers for attending training. Recipients of public assistance and other low-income people received priority for intensive and training services in local workforce investment areas in which program funds were limited. In addition, some local areas provided supportive services such as child care, transportation, and work-related financial assistance to those who qualified.
Features of the Study
The authors used a nonexperimental statistical approach called propensity-score matching to create a comparison group of people who registered at the Labor Exchange and who were similar to WIA Youth program participants in terms of demographic characteristics, including gender, age, educational attainment, race, and employment and earnings history. The authors collected Unemployment Insurance records for those who had exited a WIA Youth Program or Labor Exchange from July 2001 to June 2002 to estimate the long-term impacts of the WIA Youth Program in quarters 9 to 12 after program exit. They also collected Unemployment Insurance records for those who exited the WIA Youth Program or Labor Exchange from July 2003 to June 2004 to estimate the short-term impacts in the 3 quarters following program exit. The 1,974 WIA Youth participants who exited in 2001–2002 were matched to a sample of 782 comparison group members, which was drawn from the 24,203 youths who registered at the Labor Exchange. The 3,424 WIA Youth participants who exited in 2003–2004 were matched to a sample of 1,179 comparison group members, which was drawn from the 22,143 youths who registered at the Labor Exchange. The authors then compared the two groups on employment, quarterly hours worked, quarterly earnings, and hourly wages. Program participants were on average 17 years old; 40 percent of participants were female and about 40 percent were minorities.
Findings
Employment
- The study found that, compared with those registered for Labor Exchange services, the employment rate for the WIA Youth program group increased by 10.3 percentage points 9 to 12 quarters after program exit. Compared with those registered for Labor Exchange services, the number of hours worked per quarter for the WIA Youth Program group decreased by 13 hours 3 quarters after program exit and increased by 31.1 hours 9 to 12 quarters after program exit.
Earnings and wages
- The study found that, 3 quarters after program exit, average quarterly earnings decreased by $235; 9 to 12 quarters after program exit, those earnings increased by $288 for those who took part in the WIA Youth Program compared with those registered for Labor Exchange services. Compared with those registered for Labor Exchange services, the average hourly wage for the WIA Youth Program group increased by 68 cents 9 to 12 quarters after program exit.
Considerations for Interpreting the Findings
Although the authors accounted for many characteristics of the treatment and comparison groups in their analysis, the decision to define the groups based on their date of program exit rather than program entry is problematic. For example, if the average length of participation was 6 months for the WIA Youth group compared with one month for the Labor Exchange group and we compared the groups’ earnings 6 months after their recorded exit dates, we would see WIA Youth participants’ earnings about 12 months after they started receiving services and Labor Exchange participants’ earnings about 7 months after they started receiving services. If everyone stayed on their original upward-sloping wage trajectory, it would appear as though the WIA Youth participants earned more 6 months after their exit dates. However, this would not be attributable to receiving WIA Youth program services; it would be caused by the difference in elapsed time across the groups (12 months for WIA Youth participants versus 7 months for Labor Exchange participants). Therefore, studies defining the groups based on exit date instead of entry date without accounting for program length can receive only a low evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the treatment and comparison group were compared at different follow-up points and therefore were not equivalent. This means we are not confident that the estimated effects are attributable to WIA Youth program; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D.J. Besharov & P.H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings (pp. 347-370). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.