Absence of conflict of interest.
Citation
North Carolina State University. (2018). Third-party evaluation of DOL Round 4 TAACCCT grant Mechatronics Re-envisioned: Final evaluation report. Raleigh, NC: North Carolina State University Industry Expansion Solutions.
Highlights
- The study’s objective was to assess the impact of the Mechatronics Re-envisioned (MRE) program on education outcomes.
- Using college administrative data, the authors conducted a nonexperimental study to compare the differences in outcomes between students in the MRE program and a matched comparison group.
- The study found a significant relationship between MRE program participation and higher rates of program retention and completion, relative to the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRE program; other factors are likely to have contributed.
Intervention Examined
The Mechatronics Re-envisioned (MRE) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
A TAACCCT grant funded the Mechatronics Re-envisioned (MRE) program at Central Piedmont Community College in Charlotte, North Carolina. The MRE program’s objective was to train workers for employment in mechatronics, an area of local need. Through the MRE program, the college expanded its existing mechatronics Associate’s degree program to add web-based curricula and a fast-track certificate program. The new program curriculum was designed to reflect employer input and prepare students for local industry careers. Students received support services including a success coach who helped them with academic and career planning. In addition, some students could participate in apprenticeships as part of their training.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students in the MRE program to a matched comparison group. The treatment group included 110 students who were enrolled in an MRE course from Fall 2015 to Fall 2017. The comparison group included 392 students who took the same courses from Fall 2012 to Fall 2014, before the program enhancements were implemented. Using college administrative data, the authors assessed differences in program retention and completion between the two groups using a statistical model that controlled for differences in age, gender, and race/ethnicity.
Findings
Education and Skills Gain
- The study found a significant relationship between MRE program participation and program retention, with higher proportions of students in the treatment group retained in their program than students in the comparison group.
- The study also found that MRE program participation was significantly associated with higher rates of program completion relative to the comparison group.
Considerations for Interpreting the Findings
The authors did not account for preexisting differences between the groups before program participation. Specifically, the authors did not account for differences in financial disadvantage or baseline education outcomes. These preexisting differences between the groups—and not the MRE program—could explain the observed differences in outcomes. In addition, the authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRE program; other factors are likely to have contributed.