Absence of conflict of interest. This study was conducted by staff from ICF, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Karageorge, K., & Ferroggiaro, E. (2017). Accelerating Connections to Employment volume I final evaluation report. Fairfax, VA: ICF.
Highlights
- The study’s objective was to examine the impact of the Accelerating Connections to Employment (ACE) program on education, earnings, and employment outcomes.
- The authors used a randomized controlled trial. Eligible participants were randomly assigned to either the treatment group, which received ACE training services, or the control group, which received business-as-usual services. The primary data sources were administrative enrollment and tracking records, one- and two-year follow-up surveys, and state unemployment insurance (UI) records.
- The study found that the ACE program had a significant, positive impact on employment rates, earnings, and rates of occupational credential attainment when compared to the control group. However, individuals in the control group were significantly more likely to enroll in other training or education programs than ACE participants.
- The quality of causal evidence presented in this report is high for the education outcome measure "enrolled in additional training or education program" because it was based on a randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the ACE program, and not to other factors. However, the quality of causal evidence is moderate for the remaining education, earnings, and employment outcomes because sample attrition for these outcomes was high, but the authors controlled for key differences between the treatment and control groups at baseline. This means we are somewhat confident that estimated effects on these outcomes would be attributable to the ACE program, but other factors might have also contributed.
Intervention Examined
The Accelerating Connections to Employment (ACE) Program
Features of the Intervention
The Accelerating Connections to Employment (ACE) program was an initiative that sought to address workforce demands and the challenges of inadequate training and career paths for low-skilled job seekers. The initiative was funded by a $12 million Workforce Innovation Fund grant awarded to the Baltimore County Department of Economic and Workforce Development by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). The ACE program created a formal partnership between Workforce Investment Boards (WIBs) and community colleges designed to improve employment and employment-related outcomes for low-skilled workers. The program was implemented at nine sites across four states (Connecticut, Georgia, Maryland and Texas) from 2012 to 2016. The sites represented a mix of urban, rural and suburban communities. ACE combined evidence-based education and training services within the workforce system to help low-skilled individuals build their career paths. The ACE model was comprised of several components which included dual instruction, contextualized learning, community engagements, credentials, campus involvement, student support services, and career navigation. Participants in the ACE program received integrated basic skills and vocational training, co-teaching, career navigation, job development, and support services.
Features of the Study
The authors used a randomized controlled trial to compare the outcomes of the treatment group to the control group across the nine sites. Participants were recruited by either the community college, the One-Stop center, or both. After recruitment, individuals received information about the ACE program and other available services and were given assessments to determine their eligibility. Basic eligibility criteria were the same across all sites and included whether participants had (1) basic skills to benefit from training, (2) basic language proficiency to participate in training, and (3) no other significant factors that would affect the potential benefits of training. Some sites also required background checks and immunization records in order to determine eligibility to gain employment for a specific field of training. Eligible individuals were randomly assigned to the treatment or control group.
The treatment group consisted of 1,258 participants who were enrolled in the ACE program and received training services. The control group was comprised of 910 participants who were provided alternative services (GED or adult education programs, alternative education programs, job placement assistance, and support services) available through each WIB or community college. Data sources included administrative enrollment and tracking records housed in the Effort to Outcomes (ETO®) data management system, one- and two-year follow-up surveys, and state unemployment insurance (UI) records. The authors used statistical models to compare education, earnings, and employment outcomes between the treatment and control groups, controlling for gender, age, race, ethnicity, standardized test scores, level of formal education, and employment status at enrollment.
Study Sites
- Anne Arundel County, Maryland
- Atlanta, Georgia
- Austin, Texas
- Baltimore City, Maryland
- Baltimore County, Maryland
- Montgomery County, Maryland
- New Haven, Connecticut
- Prince George’s County, Maryland
- Upper Shore, Maryland
Findings
Education and skills gains
- The study found that participants in the ACE program held significantly more vocational, technical, or professional certificates than control group members both one (53% vs. 35%) and two years (60% vs. 38%) after training. However, individuals in the control group were significantly more likely to enroll in other training or education programs one year after training than ACE participants (22% vs. 11%).
- The study did not find any relationship between ACE program participation and holding a GED or high school diploma.
Earnings and wages
- The study found that ACE program participants had higher earnings than the control group, earning between $1,300 and $11,200 more, on average. In Connecticut, the differences were significant only eight quarters after the program end date while the differences were significant both four and eight quarters after the program end date in Maryland and Texas. No significant differences in earnings were found between the groups in Georgia.
- The study did not find a significant impact between participation in the ACE program and self-reported hourly wages.
Employment
- The study found significantly higher rates of employment in the treatment group than the control group for Texas and Maryland, both one (82% vs. 69%) and two years (79% vs. 65%) after training.
- The study also found a significant difference in self-reported employment two years after training, with ACE participants reporting higher rates of employment (82%) than the control group (63%).
- The study found no significant impact of participation in the ACE program on the amount of time it took to find a job.
- The study found no significant relationship between ACE program participation and the likelihood of receiving a promotion or a raise.
Considerations for Interpreting the Findings
Although the study design was a randomized controlled trial, the study had high attrition for the earnings and employment outcomes, and all but one education outcome. However, the authors demonstrated that the study groups included in the analysis had equivalent baseline characteristics. Therefore, the study receives a moderate causal evidence rating for those outcomes. Additionally, the authors note that sites were allowed to randomize more study participants into treatment than control to meet enrollment goals. The authors included adjusted weights in the analyses.
Causal Evidence Rating
The quality of causal evidence presented in this report is high for the education outcome measure "enrolled in additional training or education program" because it was based on a randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the ACE program, and not to other factors. However, the quality of causal evidence is moderate for the remaining education, earnings, and employment outcomes because sample attrition for these outcomes was high, but the authors controlled for key differences between the treatment and control groups at baseline. This means we are somewhat confident that estimated effects on these outcomes would be attributable to the ACE program, but other factors might have also contributed.
Additional Sources
Modicamore, D., Lamb, Y., Taylor, J., Takyi-Laryea, A., Karageorge, K., & Ferroggiaro, E. (2017). Accelerating Connections to Employment volume II appendices. Fairfax, VA: ICF.