Absence of conflict of interest.
Citation
Cosgrove, J., & Cosgrove, M. (2018). Third-party evaluation of MoSTEMWINs: Implementation, outcomes, and impact. St. Louis, MO: Cosgrove & Associates, LLC.
Highlights
- The study’s objective was to examine the impact of the Missouri STEM Workforce Innovation Networks (MoSTEMWINs) program on education and employment outcomes.
- The authors used a nonexperimental design to compare education and employment outcomes of MoSTEMWINs participants to a historical comparison group.
- The study found that MoSTEMWINs participants were significantly more likely to complete the program and attain employment than the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MoSTEMWINs program; other factors are likely to have contributed.
Intervention Examined
MoSTEMWINs
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2014, a consortium of 12 Missouri community colleges and the state's technical college was formed under the Missouri Community College Association. Missouri's STEM Workforce Innovation Networks program (MoSTEMWINs) was a TAACCCT funded program that provided training for STEM related careers. The MoSTEMWINs program, targeting adult learners, long-term unemployed, displaced workers, TAA eligible, and veterans, sought to address barriers to entering and completing STEM training within the target populations. The primary goals of the program were accelerated entry into career programs, clear paths to STEM careers, and improved employment attainment. MoSTEMWINs forty instructional programs of study provided basic academic skills, contextualized learning, industry-recognized credentials, stackable credentials, and student support services.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who participated in the MoSTEMWINs program to students who did not participate. The authors compared MoSTEMWINs program participants with a historical sample of participants in non-MoSTEMWINs programs at colleges within the consortium. The sample for analysis included 1,459 first-time college students enrolled in credit bearing programs (551 MoSTEMWINs participants and 908 comparison group participants). Data sources included college databases, a statewide ETO database, Missouri Division of Workforce Development, and participant follow-up surveys. Statistical analyses were used to examine impacts on program completion and attainment of employment following program completion.
Findings
Education and skills gain
- The study found that MoSTEMWINs participation was significantly associated with program completion, with program participants being six times more likely to complete their program of study than the comparison group.
Employment
- The study found that MoSTEMWINs participation was significantly associated with employment following program completion, with program participants being 18 times more likely to attain employment than the comparison group.
Considerations for Interpreting the Findings
The authors did not account for preexisting differences between the groups before participation, such as participants’ degree of financial disadvantage. These preexisting differences between the groups—and not the MoSTEMWINs program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MoSTEMWINs program; other factors are likely to have contributed.