Absence of conflict of interest.
Citation
Jensen, J., Horohov, J., & Waddington, J. (2017). Project IMPACT: Innovative multi-industry partnership and career training project final evaluation report. Gateway Community Technical College. Lexington, KY: University of Kentucky, College of Education, Educational Policy Studies & Evaluation.
Highlights
- The study’s objective was to assess the impact of Project IMPACT on education, earnings, and employment outcomes.
- Using academic records from the college as well as data obtained from the Kentucky Center for Workforce Statistics, the authors used a nonexperimental design to compare outcomes of Project IMPACT participants to outcomes of historical cohort students who were enrolled in the Project IMPACT programs of study prior to grant implementation.
- The study found that participating in Project IMPACT was significantly associated with a higher likelihood to take courses, pass courses, earn credits, and receive credentials. The study also found that participating in Project IMPACT was significantly associated with higher earnings in the quarter following enrollment.
- The quality of causal evidence presented in this study is low for the education and employment outcomes because the authors used a comparison group from previous enrollment years presenting a confounding factor and because the authors did not ensure the groups being compared were similar before the intervention, respectively. This means we are not confident that the estimated effects are attributable to Project IMPACT; other factors are likely to have contributed.
Intervention Examined
Project IMPACT
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2013, Gateway Community and Technical College (GCTC) in Kentucky received a Trade Adjustment Assistance Community College Career Training (TAACCCT) grant to implement the Innovative Multi-Industry Partnership and Career Training (IMPACT) program. The purpose of the IMPACT project was to enhance career preparation in logistics, manufacturing, heating and cooling, and energy fields. The project provided a combination of academic and career coaching in cooperation with workforce development partners, community agencies, and industry employers. The program targeted eligible workers, veterans and underemployed adults who were pursuing careers in logistics, manufacturing, energy, and heating and cooling fields. Key components of Project IMPACT included academic and career coaching in partnership with workforce development partners, community agencies, and industry employers.
Features of the Study
The study was conducted at Gateway Community Technical College in Florence, Kentucky. The authors compared outcomes of Project IMPACT participants to outcomes of historical cohort students who were enrolled in the Project IMPACT programs of study before implementation of the grant. The authors used a nonexperimental design with propensity score matching to ensure participants in both groups were similar on baseline characteristics. After the matching procedure, there were 96 students who had received the full Project IMPACT program treatment, and there were 93 students in the comparison group.
Findings
Education
- The study found that participating in Project IMPACT was significantly associated with taking and passing more courses. Project IMPACT students took an average of 20.4 courses and passed 18.2 of them; students in the comparison group took an average of 5.0 courses and passed 3.2.
- The study found that participating in Project IMPACT was significantly associated with earning more credits, with higher proportions of Project IMPACT students earning credits (93%) than comparison students (73%).
- The study found that participating in Project IMPACT was significantly associated with earning more credentials, with higher proportions of Project IMPACT students earning credentials (62%) than comparison students (15%).
Employment
- The study did not find a significant relationship between Project IMPACT and employment rates in the quarter after enrollment.
Earnings
- The study found that participating in Project IMPACT was significantly associated with an increase in earnings in the quarter after enrollment. Project IMPACT students had nearly double the mean quarterly wages as the comparison students ($9,100 and $4,413, respectively).
Considerations for Interpreting the Findings
The authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the economy) and not the intervention. Additionally, the authors created a matched group of a historical cohort to compare to Project IMPACT students. However, for the employment outcomes, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as a pre-intervention measure of employment. These preexisting differences between the groups—and not Project IMPACT —could explain the observed differences in outcomes. Finally, the authors also note that employment data was only available for students one month after completion and therefore, students who were employed and also still enrolled in the program, were not included in the sample. They also note that the location of the program on the border of Kentucky and Ohio may have misrepresented the employment data as individuals who found work in Ohio would not have been captured within the Kentucky data (although most of the students did have Kentucky addresses).
Causal Evidence Rating
The quality of causal evidence presented in this study is low for the education and employment outcomes because the authors used a comparison group from previous enrollment years presenting a confounding factor and because the authors did not ensure the groups being compared were similar before the intervention, respectively. This means we are not confident that the estimated effects are attributable to Project IMPACT; other factors are likely to have contributed.