Citation
Hollenbeck, K. (2009). Return on investment analysis of a selected set of workforce system programs in Indiana. Indianapolis, IN: Indiana Chamber of Commerce Foundation. [TAA]
Highlights
- The study’s objective was to examine the impact of the Trade Adjustment Assistance (TAA) Program on the employment, earnings, and benefit receipt of dislocated workers in Indiana.
- The author used a nonexperimental method to compare the short-term (three quarters after program exit) and long-term (seven quarters after program exit) employment and earnings between those who took part in the TAA Program relative to those who participated in the WorkOne program.
- The study found that, compared with those who participated in the WorkOne program, TAA Program participants had higher employment and lower benefit receipt.
- The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the TAA Program; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Trade Adjustment Assistance (TAA) Program
Features of the Intervention
The TAA Program was designed to facilitate training and provide support to workers who have lost their jobs as a result of foreign competition. The TAA Program served people ages 18 and older through core services; these included job training, job search, relocation support, and assistance with healthcare premium costs. Receipt of income assistance and other benefits varied by state and nature of foreign competition.
Features of the Study
The author used a nonexperimental statistical approach called propensity score matching to create a comparison group of people who participated in WorkOne and were similar to TAA Program participants in terms of demographic characteristics, including gender, age, educational attainment, race, and employment and earnings history. The author then compared the two groups on employment, quarterly earnings, and benefit receipt. The author collected Indiana Workforce Intelligence System records and Unemployment Insurance records for those who had exited the TAA or WorkOne program in fiscal year 2006. The study included a sample of 2,376 workers who received services through the TAA Program and 272,780 workers who received services through the WorkOne program in Indiana.
Findings
- Employment. The study found that, compared with that of those who participated in WorkOne, the employment rate for the TAA group significantly increased by 5.1 percentage points in the seventh quarter after program exit.
- Benefit receipt. The study found that average quarterly Unemployment Insurance receipt significantly decreased by $95 in the third quarter after program exit for those who took part in the TAA program compared with those in the WorkOne group.
Considerations for Interpreting the Findings
Although the author accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the author’s decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the TAA and WorkOne participants were on identical wage trajectories before receiving services from their respective programs and that the average length of participation in TAA services was six months, whereas that for WorkOne was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would observe TAA participants’ earnings about 12 months after they started receiving services and WorkOne participants’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the TAA participants earned more 6 months after their exit dates. However, this would not be attributable to receiving TAA services; it would be caused by the different elapsed time across the groups (12 months for TAA participants versus 7 months for WorkOne participants). Therefore, studies defining the groups based on exit date, rather than entry date, cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we would not be confident that the estimated effects would be attributable to the TAA Program; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). Short-term net impact estimates and rates of return. In D. J. Besharov & P. H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings, (pp. 347-370). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.