Citation
Hollenbeck, K. (2009). Return on investment analysis of a selected set of workforce system programs in Indiana. Indianapolis, IN: Indiana Chamber of Commerce Foundation. [WIA Youth]
Highlights
- The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Youth Program on the employment and earnings of low-income youths in Indiana State.
- The author used a nonexperimental method to compare the short-term (three quarters after program exit) and long-term (seven quarters after program exit) employment and earnings between youth who took part in the WIA Youth Program relative to youth who participated in the WorkOne program.
- The study found no statistically significant relationships between WIA Youth Program participants and employment and earnings.
- The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we would not be confident that any estimated effects are attributable to the WIA Youth Program; however, the study did not find statistically significant effects.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Workforce Investment Act (WIA) Youth Program
Features of the Intervention
The WIA Youth Program was authorized by Title I of the Workforce Investment Act of 1998 and was superseded by the Workforce Innovation and Opportunity Act (WIOA), effective in July 2015. The Youth Program services, which remained essentially the same under WIOA, were designed to provide quality employment and training services to eligible workers. Administered through local workforce investment areas, the Youth Program served people 14 to 21 years old through core services; these included job placement assistance, skills assessments, and provision of information on the labor market. In addition, those unable to obtain a job through core services alone could receive intensive services—which included counseling and specialized assessments—and vouchers for attending training. Recipients of public assistance and other low-income people received priority for intensive and training services in local workforce investment areas in which program funds were limited. In addition, some local areas provided supportive services such as child care, transportation, and work-related financial assistance to those who qualified.
Features of the Study
The author used a nonexperimental statistical approach called propensity score matching to create a comparison group of people who participated in WorkOne and were similar to WIA Youth Program participants in terms of demographic characteristics, including gender, age, educational attainment, race, and employment and earnings history. The author then compared the two groups on employment and quarterly earnings. The author collected Indiana Workforce Intelligence System records for those who had exited the WIA Youth or WorkOne program in fiscal year 2006. The study included a sample of 1,774 workers who received services through the WIA Youth Program and 272,780 workers who received services through the WorkOne program in Indiana.
Findings
- The study found no statistically significant relationships between WIA Youth Program participants and employment and earnings.
Considerations for Interpreting the Findings
Although the author accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the author’s decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the WIA Youth and WorkOne participants were on identical wage trajectories before receiving services from their respective programs and that the average length of participation in WIA Youth services was six months, whereas that for WorkOne was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would observe WIA Youth participants’ earnings about 12 months after they started receiving services and WorkOne participants’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the WIA Youth participants earned more 6 months after their exit dates. However, this would not be attributable to receiving WIA Youth services; it would be caused by the different elapsed time across the groups (12 months for WIA Youth participants versus 7 months for WorkOne participants). Therefore, studies defining the groups based on exit date, rather than entry date, cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we would not be confident that the estimated effects would be attributable to the WIA Youth Program; however, the study did not find statistically significant effects.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D. J. Besharov & P. H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings, (pp. 347-370). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.