Citation
Perez-Johnson, I., Moore, Q., & Santillano, R. (2011). Long-term findings from an experimental evaluation of three service delivery models. Series: ETAOP 2012-06. Washington, DC: Employment and Training Administration, U.S. Department of Labor.
Highlights
- The study’s objective was to examine the impact of individual training account (ITA) programs on employment, earnings, education and training, and benefit receipt.
- The authors conducted a randomized controlled trial. They used Unemployment Insurance (UI) records of the full sample and a follow-up survey conducted on average seven years after random assignment for a random subsample of the full sample (4,800 of 7,920) to determine education and training, employment, and earnings outcomes.
- The study found that a significantly higher percentage of the maximum customer choice group (59 percent) had earned a certificate or degree from a training program within 3 years of random assignment compared with the guided customer choice group (53 percent).
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the ITA models, and not to other factors.
Intervention Examined
The Individual Training Accounts (ITA) models
Features of the Intervention
The study examined dislocated workers and other adults who were new customers of the participating Workforce Investment Area (WIA) agencies who were eligible for ITAs. WIAs have flexibility in how they implement ITAs, which customers use to fund training programs of their choice from a wide selection of state-approved providers. The ITA experiment tested three alternative ways to manage customers’ choices of training programs: structured customer choice, guided customer choice, and maximum customer choice.
Dislocated workers and adults (18 years of age and older) eligible for WIA training assigned to the structured customer choice group received intensive, mandatory weekly counseling. Counselors directed clients to training selections that maximized return on investment and could reject clients’ choices that did not do so. Workers in this condition could also use an ITA to cover the full cost of training (up to $8,000). Those assigned to the guided customer choice group received less intensive but still mandatory weekly counseling that guided them to appropriate training selections. Counselors could not reject clients’ choices under this condition. Those in the guided customer choice group also received a fixed ITA established for their area ($3,000 to $5,000). Those assigned to the maximum customer choice group did not automatically receive, but could request, counseling to guide them to appropriate training selections and received a fixed ITA established for their area ($3,000 to $5,000). Under each condition, regional workforce boards and human services departments provided program services.
Features of the Study
The study examined dislocated workers and other adults who were new customers of the participating WIA agencies who were eligible for ITAs. To be eligible, customers had to be determined as in need of training, have the skills and qualifications to complete training, have received at least one core and one intensive service, be unable to obtain funding assistance to pay for training from other sources, and consent to participate in the study. The authors randomly assigned 7,920 customers to one of the three models tested. They used UI records of the full sample and a follow-up survey conducted on average seven years after random assignment for a random subsample of the full sample (4,800 of 7,920) to determine education and training, employment, and earnings outcomes. The authors estimated impacts for each site using statistical models that accounted for group assignment and participants’ baseline characteristics.
Study Sites
- Phoenix, Arizona
- Maricopa County, Arizona
- Bridgeport, Connecticut
- Jacksonville, Florida
- Atlanta, Georgia
- Northeast Georgia
- North Cook County, Illinois
- Charlotte, North Carolina
Findings
- The study found that 59 percent of the maximum customer choice group had earned a certificate or degree from a training program within three years of random assignment, which was significantly higher than the 53 percent of the guided customer choice group that had earned a certificate or degree. This finding was significant at the 5 percent level.
Considerations for Interpreting the Findings
Outcomes based on data that did not focus on job spells are rated high because the study is a randomized controlled trial and attrition was low. For the employment and earnings outcomes, the authors did not establish that attrition was low or that the groups were similar in their characteristics before the intervention began. As a result, these outcomes received a low evidence rating and are not reported here.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the ITA model intervention, and not to other factors.