Skip to main content

PACE cross-program implementation and impact study findings (Gardiner & Juras 2019)

Absence of conflict of interest

This study was conducted by staff from Abt Associates, which administers CLEAR. The review of this study was conducted by ICF Incorporated, which also administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.

Citation

Gardiner, K., & Juras, R. (2019). PACE Cross-Program Implementation and Impact Study Findings (Report No. 2019-32). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. [WTAC]

Highlights

  • The study's objective was to examine the impact of Workforce Training Academy Connect (WTAC) on education outcomes. The authors investigated similar research questions for other programs, the profiles of which can be found here.
  • The study used a randomized controlled trial to examine the impact of WTAC on education outcomes. Using participant surveys, the authors conducted a statistical model to compare the outcomes of the treatment and control group members.
  • The study did not find a significant difference between WTAC participants and control participants in the likelihood of earning one or more credential(s).
  • This study receives a high evidence rating. This means we would be confident that any estimated effects would be attributable to Workforce Training Academy Connect (WTAC), and not to other factors. However, the study did not find statistically significant effects.

Intervention Examined

Workforce Training Academy Connect (WTAC)

Features of the Intervention

The Workforce Training Academy Connect (WTAC) at Des Moines Area Community College focused on basic skills remediation. WTAC targeted individuals with reading and math skills that were below a ninth grade level. WTAC provided accelerated online basic skills training followed by occupational training in growing employment sectors (e.g., healthcare, advanced manufacturing, and administrative support). Both phases of the program were free of charge. WTAC provided advising and non-academic supports (e.g., bus passes) during the program and employment assistance upon program completion. The program served participants with low occupational skills who also had low incomes or other barriers to employment (e.g., limited work histories, criminal justice involvement).

Features of the Study

This study was part of the multi-program Pathways for Advancing Careers and Education (PACE) evaluation. The PACE evaluation was intended to address the gap in research on the effectiveness of career pathways programs, or programs providing postsecondary training or education in growing employment sectors. This profile focuses on WTAC.

The study used a randomized controlled trial to examine the impact of WTAC. Of the 943 participants who enrolled in the study between April 2012 and December 2014, 470 were randomly assigned to WTAC (treatment group) and 473 were assigned to the control group. Control group members could not receive WTAC services but could choose to participate in other employment services within the community. The study sample was 63 percent female, 47 percent Black/African American, 15 percent Hispanic/Latino, and 34 percent White. Thirty-seven percent of the sample had a high school degree or equivalency credential without further education/training. The data for the credential receipt outcome were drawn from the PACE 18-month survey. The authors used a statistical model to compare the outcomes of the treatment group with those of the control group.

Findings

Education and skills gains

  • The study found no significant difference between treatment participants and control participants in the likelihood of earning one or more credential(s).

Considerations for Interpreting the Findings

The study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.

Causal Evidence Rating

The quality of causal evidence presented in this report is high, because it was based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects would be attributable to WTAC and not to other factors. However, the study did not find statistically significant effects.

Additional Sources

Hamadyk J., & Zeidenberg, M. (2018). Des Moines Area Community College Workforce Training Academy Connect Program: Implementation and Early Impact Report (Report No. 2018-82). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Reviewed by CLEAR

June 2022

Topic Area

Topic Area