Absence of conflict of interest.
Citation
Swan, B., Clarke, M. H., Serpa, A., DeStefano, C., & Hahs-Vaughn (2018). M-PATH: Advanced manufacturing final evaluation report. (Rep. No. 33VCDOLTACT2014.Y4S). Orlando, FL: University of Central Florida, Program Evaluation and Educational Research Group (PEER).
Highlights
- The study’s objective was to examine the impact of the M-PATH: Advanced Manufacturing Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant program on education, earnings, and employment outcomes.
- Using a nonexperimental design, the authors compared students in the M-PATH TAACCCT program to a matched comparison group.
- The study found that participation in the M-PATH TAACCCT program was significantly related to increased program completion and decreased continued education in another certificate program. The study also found that non-incumbent workers who completed the program were significantly more likely to enter employment than the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not include sufficient control variables. This means we are not confident that the estimated effects are attributable to the M-PATH TAACCCT program; other factors are likely to have contributed.
Intervention Examined
The M-PATH: Advanced Manufacturing Trade Adjustment Assistance Community College and Career Training (TAACCCT) Grant Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Valencia College received a TAACCCT grant for the M-PATH: Advanced Manufacturing (M-PATH) program to offer additional specialized advanced manufacturing jobs. The goals of the M-PATH program were to increase attainment of certifications, certificates, and relevant credentials for workers in "growth industry sectors"; provide innovative curriculum that meets industry needs and increases learning and retention for workers; and show that program participants improve employment outcomes as a result of the M-PATH program. The program was designed to provide TAA-eligible workers and other adults for employment in the targeted industry sectors. The College partnered with regional manufacturing, CareerSource (local workforce investment board), and other industry relevant organizations to develop the educational pathway program which included for-credit programs as well as continuing education. In the four years of the grant, 14 programs were developed or improved to include 14 that provided industry relevant certificates. The programs included 10 "flexible" labs to support hands-on training and instruction. Finally, the program included student-tailored support services and advising, career coaching, non-traditional recruitment methods, and tutoring to students.
Features of the Study
The study took place at Valencia College and used a nonexperimental design to compare the outcomes of students who participated in the M-PATH TAACCCT program to students who did not participate. The comparison group included students who were enrolled or formerly enrolled in a similar program of study. The authors matched M-PATH students to similar students using propensity scores developed from demographic variables. Study participants included 147 students in the treatment group and 244 students in the comparison group. Using data from institutional records and Unemployment Insurance data from the local workforce investment board, the authors conducted statistical tests to examine differences in outcomes between the groups.
Findings
Education and skills gain
- The study found that students in the M-PATH TAACCCT program were significantly more likely than comparison group students to complete their program of study (4,000 times more likely).
- However, the study found that students in the M-PATH TAACCCT program were significantly less likely than comparison group students to continue education in another certificate program (50 times less likely).
Earnings and wages
- The study did not find a significant relationship between participation in the M-PATH TAACCCT program and receipt of a wage increase.
Employment
- The study found that non-incumbent workers who completed the program were significantly more likely to enter employment than the comparison group (26 times more likely).
Considerations for Interpreting the Findings
The authors created a matched group of comparison students to compare to M-PATH students. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage which is required by the review protocol. These preexisting differences between the groups—and not the M-PATH TAACCCT program—could explain the observed differences in outcomes. Additionally, the authors used a single regression to impute data for missing data on some of their control variables. The authors also note that 40% of students were enrolled in the M-PATH program before students enrolled in the comparison group, which could have affected the outcome of program completion. Finally, some of the comparisons had no or few students in some of the classifications, and the authors state that this could have affected the accuracy and generalizability of the study as odds ratios may be inflated. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not include sufficient control variables. This means we are not confident that the estimated effects are attributable to M-PATH TAACCCT program; other factors are likely to have contributed.