Citation
Hollenbeck, K., & Huang, W-J. (2008). Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [CTE]
Highlights
- The study’s objective was to examine the impact of the Carl Perkins Postsecondary Career and Technical Education (CTE) program on employment and credential completion.
- The authors used administrative records to compare the outcomes of low-income adults who took part in the CTE program with the outcomes of a nonexperimental matched group of adults who did not take part in the program.
- The study found that CTE participants had higher employment and credential completion rates compared with those of people who did not participate in the program.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to CTE; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Workforce Investment Act (WIA) Carl Perkins Postsecondary Career and Technical Education (CTE) Program
Features of the Intervention
The postsecondary CTE program provides career-focused education though Virginia’s community college system. CTE students take courses and work towards credentials that will prepare them for specific occupational fields.
Features of the Study
Using data from program administrative records, Unemployment Insurance records, the Wage Record Interstate System, and the community college system, the authors compared education and employment outcomes of those who took part in the Carl Perkins Postsecondary CTE program with outcomes of those who did not take part in the program (both groups participated in an employment services program). The authors compared the two groups on employment two and four quarters after program exit, and on the percentage of each group that had earned an educational credential during the program or within one year of program exit. This analysis included 30,058 CTE participants in Virginia who exited the program from July 2004 to June 2005.
Findings
- Employment. The study found that CTE participants were significantly more likely to be employed than members of the comparison group, with an employment rate that was 1.0 percentage point higher in the second quarter and 1.9 percentage points higher in the fourth quarter after program exit.
- Education and/or training attainment and completion: The study found that CTE participants were significantly more likely than members of the comparison group to have earned an educational credential during the program or within one year of program exit (a difference of 86.3 percentage points).
Considerations for Interpreting the Findings
The authors present unadjusted treatment effects in the study. This profile reports adjusted effects obtained directly from the author.
Although the authors accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the authors’ decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the CTE participants and comparison group members were on identical wage trajectories before receiving services from their respective programs and that the average length of participation in CTE services was six months, whereas that for the employment services program was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would observe CTE participants’ earnings about 12 months after they started receiving services and comparison group members’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the CTE participants earned more 6 months after their exit dates. However, this would not be attributable to receiving CTE services; it would be caused by the different elapsed time across the groups (12 months for CTE participants versus 7 months for comparison group members). Therefore, studies defining the groups based on exit date, rather than entry date, cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to CTE; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D.J. Besharov & P.H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings (pp. 347-370). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.