Absence of conflict of interest.
Citation
Cosgrove, J. J., Cosgrove, M. S., & Bragg, D. D. (2015). Evaluation of MoHealthWINS outcomes and impact. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/5125/SEPTEMBER%202015%20FINAL%20VERSION%20OF%20MHW%20OUTCOME%20IMPACT%20EVALUATION%20REPORT.pdf?sequence=1&isAllowed=y
Highlights
- The study’s objective was to assess the relationship between the MoHealthWINS program and education and employment outcomes.
- The study used a nonexperimental design, where the program participants were compared to a matched comparison group. Using college administrative data, state employment data, and employer data, the authors compared the program completion and employment rates between the treatment and comparison group.
- The study found a significant relationship between program participation and the likelihood to complete the program and be employed post-program completion.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to MoHealthWINS program; other factors are likely to have contributed.
Intervention Examined
MoHealthWINS
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
TAACCCT funded a program called MoHealthWINS that provided training for healthcare careers. The Missouri Community College Association (MCCA) led the project and implemented it in the 12 community colleges in Missouri and in one technical college in the state. The program targeted populations that were under-skilled, unemployed, veterans, and TAA-eligible, however, these characteristics were not required for participation in the program. During the grant, the colleges revised and created new healthcare training programs, with a focus on those that were non-credit. The programs offered stacked credentials designed to be relevant to jobs with local employers, and students could receive support services. Most colleges implementing the program offered contextualized instruction, online learning, and accelerated, modular courses.
Features of the Study
The study used a nonexperimental design that compared outcomes between the treatment group and a matched comparison group. The impact analyses included students who were in for-credit programs. Students in the treatment group were enrolled from 2011 to 2014, and a comparison group enrolled in Fall 2009. The study included 4,251 students in the treatment group, and 1,704 in the comparison group across the 13 colleges. The authors used college administrative data, state employment data, and obtained information from employers to augment missing employment data. The authors compared education and employment outcomes between the treatment and comparison groups, using a statistical model controlling for gender, age, race, and employment status when starting the program, and whether they had at least some college experience before starting the program.
Findings
Education and skills gains
- The study found a significant relationship between MoHealthWINS participation and program completion, with a higher percentage of MoHealthWINS participants completing the program (66%) versus students in the comparison group (21%).
Employment
- The study found a significant relationship between MoHealthWINS participation and employment, with a higher percentage of MoHealthWINS participants being employed after the program (80%) than students in the comparison group (47%).
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to MoHealthWINS participants. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not the MoHealthWINS program—could explain the observed differences in outcomes. Additionally, the authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the economy) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to MoHealthWINS; other factors are likely to have contributed.