Citation
Plesca, M., & Smith, J. (2007). Evaluating multi-treatment programs: Theory and evidence from the U.S. Job Training Partnership Act experiment. Empirical Economics, 32, 491-528.
Highlights
- The study’s objective was to examine the impact of the Job Training Partnership Act (JTPA) on earnings and employment for disadvantaged adults.
- The authors conducted a randomized controlled trial. They analyzed data from one baseline survey, two follow-up surveys, and earnings and employment records.
- The study found that men in the treatment group had earned more at the 30-month follow-up than men in the control group, and that women in the treatment group experienced significantly better earnings and employment outcomes at several points compared to the control group.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to JTPA, and not to other factors.
Intervention Examined
Job Training Partnership Act
Features of the Intervention
JTPA was created to improve the labor outcomes of youth and disadvantaged adults. This study explored the impact of three “treatment streams” under the JTPA: (1) free classroom training in occupational skills; (2) subsidized on-the-job training with private employers; and (3) a residual “other” category that includes those referred to job search assistance, adult basic education, subsidized work experience with public or not-for-profit employers, or other JTPA assistance not including classroom occupational skills or on-the-job training (unless both were recommended). Individuals were eligible to participate in JTPA if they fell below an income threshold or if they received food stamps, Aid to Families with Dependent Children, or Temporary Assistance for Needy Families.
Features of the Study
The study included 14,911 individuals over the age of 21 who applied for JTPA assistance at one of the 16 participating locations across the United States from November 1987 to September 1989, were deemed eligible to participate, and were recommended to receive at least one JTPA service. Study participants were placed into one of three treatment streams based on caseworkers’ recommendations and then randomly assigned to treatment in that stream or to a control condition with no access to JTPA resources for 18 months. The authors used surveys administered at baseline, at 18 months after assignment, and at 30 months after assignment to compare earnings and employment outcomes of each of the treatment arms, as well as all treatment groups combined, to the earnings and employment outcomes of control group members. The 30-month surveys were supplemented with administrative earnings and employment records in 12 of the 16 sites.
Findings
- Employment. The study found that women who were randomly assigned to any JTPA treatment arm were 3 to 4 percentage points more likely to be employed 18 and 30 months after program enrollment than women in the control group. These differences were statistically significant.
- Earnings. Men and women randomly assigned to receive any JTPA service earned approximately $1,200 more in the first 30 months after program enrollment than those in the control group. These differences were statistically significant.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to employment and earnings. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated. This review limited the number of comparisons presented by examining only those outcomes for which the authors found statistically significant differences in the impacts across the JTPA treatment arms.
In addition, not all treatment group members enrolled in JTPA, so this analysis examines access to—not receipt of—JTPA services. Just over half of study participants (56 percent of men and 54 percent of women) who were assigned to any JTPA service actually received JTPA services
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to JTPA and not to other factors.
Additional Sources
Bloom, H., Orr, L., Cave, G., Bell, S., & Doolittle, F. (1993). The National JTPA Study: Title II-A impacts on earnings and employment at 18 months. Bethesda, MD: Abt Associates.
Orr, L., Bloom, H., Bell, S., Lin, W., Cave, G., & Doolittle, F. (1994). The National JTPA Study: Impacts, benefits, and costs of Title II-A. Bethesda, MD: Abt Associates.