Citation
Miller, C., Bos, J., Porter, K., Tseng, F., & Abe, Y. (2005). The challenge of repeating success in a changing world: Final report on the Center for Employment Training replication sites. New York: MDRC.
Highlights
- This report’s objective was to examine the effectiveness of the Center for Employment Training (CET) model. Under this model, local employers help design training programs and provide them in a work-like environment to economically disadvantaged youth who are not in school or employed. A separate report examined CET implementation.
- The effectiveness of the CET model in improving employment, earnings, and participation in education and training programs was evaluated using a randomized controlled trial (RCT) conducted in 12 sites in seven states.
- The study found few statistically significant impacts on outcomes measured over a 54-month follow-up period.
- The quality of the causal evidence presented in this study is moderate. This means we have confidence that the effects estimated in this study are attributable at least in part to the CET program. However, other factors not accounted for in the study might also have contributed to the estimated effect.
Intervention Examined
The Center for Employment Training (CET) Replication
Features of the Intervention
The CET program targets disadvantaged, out-of-school youth ages 16 to 21. It was first implemented in a site in San Jose, California. Two RCTs conducted in the late 1980s confirmed that CET-San Jose produced positive results for participating youths. Therefore, the U.S. Department of Labor initiated a replication study to discover whether the CET-San Jose model could be implemented successfully in other settings. The CET replication sites agreed to participate in the replication study, and the national CET office in San Jose provided each site with technical assistance.
The CET model has four core features:
- It provides occupational and basic skills training in a work-like setting, and trainees advance at their own pace.
- It requires a full-time commitment from students, to accustom them to a work schedule.
- It maintains close connections to employers, who help design and implement the training programs.
- It has an extensive orientation process, during which the rigor of the program is emphasized; less motivated individuals may drop out at that point.
Features of the Study
The study was an RCT conducted in 12 sites across the country. From 1995 to 1999, the study authors randomly assigned 748 youth to a treatment group eligible to receive CET services and 737 youth to a control group ineligible to receive CET services for 24 months. The authors estimated the effectiveness of the CET program by comparing outcomes of youth in the CET group to outcomes of youth in the control group. The outcomes of interest were employment, earnings, and participation in education and training programs measured up to 54 months after random assignment.
Study Sites
- El Centro, California
- Oxnard, California
- Riverside, California
- San Francisco, California
- Santa Maria, California
- Orlando, Florida
- Chicago, Illinois
- Reidsville, North Carolina
- Reno, Nevada
- Camden, New Jersey
- Newark, New Jersey
- New York, New York
Findings
- The authors reported impacts for eight earnings and employment outcomes measured less than one year after random assignment; none of these was statistically significant.
- The authors also reported impacts on 21 employment and earnings outcomes measured from 12 to 54 months after random assignment. Two of these were statistically significant: the percentage of individuals who worked 35 hours a week or more in their most recent job was 6 percentage points higher in the CET group than in the control group, and the percentage of individuals holding a total of one job during the 54-month follow-up period was 4 percentage points lower in the CET group than in the control group.
- The study also reported impacts for 80 outcomes related to participation and attainment in education and/or training programs. In general, these impacts indicated that CET group members were more likely to participate in a training program within the first year of random assignment, but this impact declined in subsequent years. In addition, CET group members were more likely than control group members to receive a training certificate.
Considerations for Interpreting the Findings
Implementing the CET model at the replication sites was challenging. The study reported that only 4 of the 12 sites implemented the model with high fidelity, 6 others implemented the program with medium fidelity, and 2 did so with low fidelity. The authors identified organizational stability as a key feature of successful programs; those that had high turnover in leadership and staff were less successful than more stable programs at sustaining the program model. Difficulty developing close relationships with employers was another challenge to successful implementation.
The study authors estimated multiple related impacts on employment, earnings, education, and training. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings from the study is likely to be overstated.
Causal Evidence Rating
The quality of the causal evidence presented in this study is moderate because, although it was an RCT with high attrition, the authors demonstrated that the treatment and control groups were equivalent on important characteristics before random assignment. Therefore, we have confidence that estimated effects are attributable at least in part to the CET program, although other factors not accounted for in the analysis could also have contributed.