Absence of conflict of interest.
Citation
Bucci, P., Evans, W.D., Friesenborg, L., Fox, D.A., Bishop, K., Balraj, N., & Jang, B. (2018). Minnesota advanced manufacturing project: Final external evaluation report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/18521/MnAMP%20Final%20Evaluation%20Report.pdf?sequence=1&isAllowed=y
Highlights
- The study’s objective was to examine the impact of the Minnesota Advanced Manufacturing Partnership (MnAMP) program on education outcomes.
- The authors used a nonexperimental design to compare education outcomes of MnAMP participants to a matched comparison group using institutional data.
- The study found that MnAMP program participation was significantly associated with earning fewer credits and fewer credentials relative to the comparison group. However, among students who attained credentials, participating in the MnAMP program was significantly associated with completing a one-year certificate in less time than comparison group students.
- The quality of causal evidence presented in this study is low because the authors used a comparison group from previous enrollment years, presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MnAMP program; other factors are likely to have contributed.
Intervention Examined
Minnesota Advanced Manufacturing Partnership (MnAMP) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2014, a consortium of 12 community and technical colleges and two university centers led by South Central College received a TAACCCT grant to fund the Minnesota Advanced Manufacturing Partnership (MnAMP) program. The goal of MnAMP was to help unemployed adults in the state, including TAA-certified workers and veterans, gain the required skills to fill available jobs in the advanced manufacturing sector. The program specifically targeted one-year certificate programs in mechatronics, machining, and welding. The MnAMP consortium, which consisted of twelve community and technical colleges and two university centers, collaborated with employers and other stakeholders to design pathways to produce more skilled workers, shorten the time needed to obtain a credential, and enable workers to learn marketable new skills and earn increased wages. Features of the model included stackable and portable curricula, competency-based education, prior learning assessments, career coaching, apprenticeships, and distance learning.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who participated in MnAMP program to those who did not. The authors matched MnAMP participants to similar nonparticipants using propensity scores developed from socio-demographic information. A total of 4,583 students from eight of the colleges in the Consortium participated in the study. The treatment group included 2,403 students who enrolled during the grant funding period (2014-2018). The comparison group included 2,180 students who enrolled in similar academic programs before the program was implemented (2010-2014). Using institutional data, the authors conducted statistical models to examine group differences in education outcomes. Outcomes included credits earned, credentials attained, or time to credential completion.
Study Sites
- Lake Superior College in Duluth, Minnesota Minnesota West
- Community and Technical College in Worthington, Minnesota
- Normandale Community College in Bloomington, Minnesota
- Northland Community & Technical College in East Grand Forks, Minnesota
- Ridgewater College in Willmar, Minnesota
- Riverland Community College in Austin, Minnesota
- Saint Paul College in Saint Paul, Minnesota
- South Central College in in Faribault, Minnesota
Findings
Education and skills gain
- The study found that MnAMP program participation was significantly associated with earning fewer credits (2 fewer) than comparison students.
- The study also found that MnAMP program participation was significantly associated with attaining fewer credentials (30% less) than comparison students.
- However, among students who completed a credential, participating in the MnAMP program was significantly associated with completing a one-year certificate in less time (1.5 months less) than comparison students.
Considerations for Interpreting the Findings
The authors compared students who participated in the programs during grant implementation (2014-2018) to a historical comparison group of students who participated in similar academic programs in the years prior to grant implementation (2010-2014). Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the program. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this study is low because the authors used a comparison group from previous enrollment years presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MnAMP program; other factors are likely to have contributed.