Absence of conflict of interest.
Citation
Corporation for a Skilled Workforce & The New Growth Group. (2016). Multi-State Advanced Manufacturing Consortium (M-SAMC) TAACCCT Round II grant final evaluation report. Ann Arbor, MI: Corporation for a Skilled Workforce (CSW); Cleveland, OH: The New Growth Group, LLC.
Highlights
- The study’s objective was to examine the impact of the Multi-State Advanced Manufacturing Consortium (M-SAMC) program participation on education outcomes. This summary focuses on the findings at Bridge Valley Community and Technical College in West Virginia.
- The authors used a nonexperimental design to compare outcomes of M-SAMC program participants with a matched comparison group.
- The study found that the odds of program completion were significantly higher for participants in the M-SAMC programs compared to participants in the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the M-SAMC; other factors are likely to have contributed.
Intervention Examined
Multi-State Advanced Manufacturing Consortium (M-SAMC)
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Multi-State Advanced Manufacturing Consortium (M-SAMC) was a consortium of 13 community colleges across 10 states designed to meet the local needs of the manufacturing industry. M-SAMC set out to develop and improve curricula, credentials, instructional design and delivery to contextualize learning, student support, student success, job placement, and partnerships with employers, community-based organizations, and unions. Competency-based education (CBE) was a major effort of the consortium including use of common industry-respected simulators and consortium-based models. All schools had access to industry standard equipment; the Integrated Manufacturing Systems Trainer (IMST), and the first level of consortium developed and delivered equipment-use training in a manufacturing industry-like instructional environment. All schools participated in at least one structured workgroup to develop the models that were later provided to all schools for adoption or adaptation based on their local context and needs.
Features of the Study
The nonexperimental study took place at Bridge Valley Community and Technical College in Kanawha County, West Virginia. The authors compared program completion among 77 M-SAMC program participants and a matched comparison group of 765 participants. Bridge Valley Community and Technical College was unable to provide a suitable group of participants for comparison in the analysis; therefore, the authors used a comparison group from another consortium school. Gadsden State Community College in Gadsden, Alabama was selected because their program types and duration were comparable to those offered at Bridge Valley. Treatment participants at Bridge Valley were enrolled between Fall 2012 and Spring 2016. The comparison group of participants were enrolled at Gadsden between the Fall 2010 and Fall 2012. The authors matched Bridge Valley participants to similar Gadsden State nonparticipants using propensity scores developed from baseline demographic information. Data sources included participant intake forms, college databases, state earning records, and post-program completion participant surveys. The authors used a statistical model to examine the odds of program completion for treatment participants versus participants in the comparison group.
Findings
Education and skills gain
- The study found that M-SAMC participation was significantly related to higher odds of program completion, where treatment participants were 5.5 times more likely to complete the program than participants in the comparison group.
Considerations for Interpreting the Findings
The authors created a matched group of non-participants to compare to M-SAMC participants. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention education/training as required by the protocol. These preexisting differences between the groups—and not the M-SAMC program—could explain the observed differences in outcomes. Additionally, the authors used a cohort from previous enrollment years at a different college as the comparison group. Because the outcome data on the treatment and comparison groups were collected from participants at different times and at different colleges, differences in outcomes could be due to time-varying factors or college varying characteristics, and not the M-SAMC program. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the M-SAMC; other factors are likely to have contributed.