Citation
Hollenbeck, K., & Huang, W.-J. (2008). Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [TAA]
Highlights
- The study’s objective was to examine the impact of the Training Adjustment Assistance (TAA) program on employment and credential completion.
- The study used administrative records to compare outcomes of low-income adults who took part in the TAA program with outcomes of a nonexperimental matched group of adults who did not take part in the program.
- The study found that TAA participants had a lower employment rate but were more likely to have obtained a training certificate compared with those who did not participate in the program.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to TAA; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Trade Adjustment Assistance (TAA) Program
Features of the Intervention
Established under the Trade Act of 1974 and amended through the Trade Adjustment Assistance Reform Act of 2002, the TAA Program was intended to provide aid to adult workers (18 and older) within an industry or group whose employment had been lost or reduced as a result of foreign trade competition. The available services included employment and case management services, job search assistance and allowances, training, relocation allowances, and some cash income supports. Eligible employees must have lost their jobs or have been notified of unemployment risk because of import competition or shifts in foreign production.
Features of the Study
Using data from program administrative records, Unemployment Insurance records, the Wage Record Interstate System, and the community college system, the authors compared education and employment outcomes of those who took part in the TAA program with outcomes of those who did not take part in the program (both groups participated in an employment services program). The authors used a nonexperimental statistical approach called propensity-score matching to compare TAA participants with similar nonparticipants. The authors compared the two groups on employment two and four quarters after program exit, and on the percentage of each group who had earned an educational credential during the program or within one year of program exit. This analysis included 2,254 TAA participants in Virginia who exited the program from July 2004 to June 2005.
Findings
- Employment. The study found that TAA participants were significantly less likely to be employed than members of the comparison group, with an employment rate that was 6.2 percentage points lower in the second quarter and 5.6 percentage points lower in the fourth quarter after program exit.
- Education and/or training attainment and completion. The study found that TAA participants were significantly more likely than members of the comparison group to earn an educational credential during the program or within one year of program exit (a difference of 67 percentage points).
Considerations for Interpreting the Findings
The authors presented unadjusted treatment effects in the study. This profile reports adjusted effects obtained directly from the authors.
Although the authors accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the authors’ decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the TAA participants and the comparison group members (who participated in the employment services program only) were on identical wage trajectories before receiving services from their respective programs, and that the average length of participation in TAA services was six months, whereas that for the employment services program was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would observe TAA participants’ earnings about 12 months after they started receiving services and comparison group members’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the TAA participants earned more 6 months after their exit dates. However, this would not be attributable to receiving TAA services; it would be caused by the different elapsed time across the groups (12 months for TAA participants versus 7 months for comparison group members). Therefore, studies defining the groups based on exit date rather than entry date cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to TAA; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D.J. Besharov & P.H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings (pp. 371–295). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.