Skip to main content

Final evaluation report Round 3 TAACCCT grant: Mission critical operations (NC State Industry Expansion Solutions 2017)

Review Guidelines

Absence of conflict of interest.

Citation

NC State Industry Expansion Solutions. (2017). Final evaluation report Round 3 TAACCCT grant: Mission critical operations. Raleigh, NC: NC State Industry Expansion Solutions.

Highlights

  • The study’s objective was to evaluate the effects of the National Consortium for Mission Critical Operations (MCO) program on educational outcomes. This summary contains the findings from Wake Tech Community College.
  • The authors used a nonexperimental design to compare the outcomes of students enrolled in the MCO program to a matched comparison group of students enrolled in the same courses of study prior to the implementation of the MCO program.
  • The study found that participation in the MCO program was significantly associated with higher rates of retention and program progress.
  • The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MCO program; other factors are likely to have contributed.

Intervention Examined

The Mission Critical Operations (MCO) Program

Features of the Intervention

The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.

The Mission Critical Operations (MCO) program was funded by the TAACCCT grant to build institutional capacity to increase credentials in a range of industries focusing on electrical/electronic, industrial, and information technology in response to the disappearance of traditional manufacturing and skilled service jobs. The MCO program had three main objectives: 1) to better prepare TAA eligible and other adults with high-skill, high wage employment or re-employment by increasing the attainment of certificates, diplomas, and other industry related credentials; 2) to improve learning outcomes and retention rates for TAA workers and other adults by redesigning curriculum with innovative and effective methods to address specific industry needs; and 3) demonstrate for TAA workers in particular improved employment outcomes. A consortium of colleges was formed to implement and monitor the MCO program. The National Consortium for Mission Critical Operations consisted of the following five colleges: Cleveland Community College, Nash Community College, Wake Technical Community College, Moultrie Technical College, and the University of North Carolina at Charlotte. Each college in the consortium implemented various intervention activities, including articulation, collaboration, course design, credential development, instruction/support, scorecard development, telepresence, and tracking/evaluation.

Features of the Study

The nonexperimental study took place at Wake Tech Community College in Raleigh, North Carolina. The authors compared the education outcomes of students who enrolled in the MCO program to students enrolled in the same courses of study prior to the implementation of the MCO program. The treatment group consisted of 404 students who participated in the MCO program after Fall 2014. The comparison group included 1,392 students enrolled between Fall 2010 and Summer 2013, in the same courses that were later impacted by the MCO funds. The authors matched MCO participants to similar nonparticipants using propensity scores developed from baseline demographic information. Using data from Wake Tech's Student Information System, the authors conducted statistical models and analyses to examine differences in education outcomes between the groups. Education outcomes included retention rates, program progress (measured as earning grades of A, B, or C in courses), and program completion.

Findings

Education and skills gain

  • The study found a significant relationship between MCO program participation and retention rates, with MCO participants approximately twice as likely to be retained than students in the comparison group.
  • The study also found a significant association between MCO program participation and higher rates of course progression than students in the comparison group.
  • However, the study did not find a significant association between MCO program participation and program completion.

Considerations for Interpreting the Findings

The authors used propensity score matching to create a comparison group; however, they did not account for preexisting differences between the groups in financial disadvantage or baseline education outcomes. These preexisting differences between the groups—and not the MCO program—could explain the observed differences in outcomes. In addition, the authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.

Causal Evidence Rating

The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MCO program; other factors are likely to have contributed.

Reviewed by CLEAR

May 2020

Topic Area