Absence of conflict of interest.
Citation
Davis, S., & Bill, N. (2018). Evaluation of the Pennsylvania’s Advanced Training and Hiring Program: Final report. Washington, DC: Impaq International.
Highlights
- The study’s objective was to assess the impact of the Pennsylvania’s Advanced Training and Hiring (PATH) initiative on education outcomes.
- Using college administrative data, the authors conducted a nonexperimental study to compare education outcomes of students in the PATH program to a matched comparison group.
- The study found that PATH participation was significantly associated with higher rates of program completion and academic progress.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention.This means we are not confident that the estimated effects are attributable to the PATH initiative; other factors are likely to have contributed.
Intervention Examined
Pennsylvania’s Advanced Training and Hiring (PATH) Initiative
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Pennsylvania’s Advanced Training and Hiring (PATH) initiative was funded by TAACCCT and operated from 2014-2018 at three community colleges in Pennsylvania: Northampton Community College (NCC), Lehigh Carbon Community College (LCCC), and Luzerne County Community College (Luzerne). The program was created to train local workers for employment within in-demand industries of manufacturing, healthcare, and logistics/transportation. To do this, the program created and revised curricula to align with local employer workforce needs, while also providing more hands-on learning opportunities and updated equipment and classroom space for instruction. The program expanded opportunities for students to earn credentials, by allowing students to stack credentials and by incorporating industry certifications into the programs. While participating in the program, students could receive various support services, such as career planning, job placement, job preparation, and referrals for other services within the college.
Features of the Study
The nonexperimental study was conducted with students from NCC and Luzerne. The authors compared the education outcomes of students who enrolled in the PATH program to students in similar academic programs at the same colleges. Study participants included 753 students in the treatment group and 1,659 students in the comparison group. The study used college administrative data to measure education outcomes in both groups. After creating a matched comparison group based on students’ age, gender, and race, the authors compared the proportion that achieved educational outcomes between the treatment and comparison groups.
Findings
Education and skills gain
- The study found a significant relationship between PATH participation and program completion, with a larger proportion of PATH students completing their program (20 percentage points more) than students in the comparison group.
- The study also found a significant relationship between PATH participation and academic progress, with a larger proportion of PATH students making normal academic progress (20 percentage points more) than students in the comparison group.
Considerations for Interpreting the Findings
The authors did not account for preexisting differences between the groups before program participation. Specifically, they did not account for differences in financial disadvantage or baseline education outcomes when creating the matched comparison group. These preexisting differences between the groups—and not the PATH initiative—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the PATH initiative; other factors are likely to have contributed.