Absence of conflict of interest.
Citation
Belville, J., Romano, G., Frisby, M., Chamberlin, M., & Strem, L. (2017). Final evaluation report: Cincinnati State Technical and Community College: Greater Cincinnati Manufacturing Careers Accelerator (GCMCA).
Highlights
- The study’s objective was to examine the impact of the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program on education outcomes.
- The author used a nonexperimental design to compare the education persistence of students in the treatment group to students in a matched comparison group.
- The study found that students in the comparison group were significantly more likely to complete the program than students in the treatment group for both the Welding or Mechanical Engineering Technology Manufacturing and Design programs.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention.This means we are not confident that the estimated effects are attributable to the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program; other factors are likely to have contributed.
Intervention Examined
Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories. Cincinnati State Technical and Community College received a grant to implement the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program, designed to support training programs on Welding and Mechanical Engineering Technology Manufacturing and Design. The program targeted TAA-eligible workers, veterans, and other individuals for employment. The programs were offered at three campuses but within two divisions: the Workforce Development Center (providing short-term workforce education and training) and the Center for Innovative Technologies (offering college credits towards an associate’s degree and certifications). The Welding program focused on student-centered learning, which included customized math and English training, adaptive learning, and two contextualized learning approaches for reading and math utilizing welding examples and materials. The community college also enhanced their Pathway to Employment Center which was designed to provide student supportive services.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who participated in the Welding and Mechanical Engineering Technology Manufacturing and Design programs to a comparison group, analyzed separately for each program. Students in the Welding group were compared with students from concurrent programs in Aviation Mechanics Airframe and Aviation Mechanics Power Plate. Students in the Mechanical Engineering Technology Manufacturing and Design program were compared to a historical comparison group of students in the same program of study prior to receiving the grant. The authors matched GCMCA participants to similar nonparticipants using propensity scores developed from demographic, education, and program information. For Welding, study participants included 51 students in the treatment group and 59 in the comparison group. For Mechanical Engineering Technology Manufacturing and Design, study participants include 122 in the treatment group and 193 in the comparison group. The outcomes included educational persistence rates (education program completion). Using student records, the authors conducted chi-square analyses and statistical models with controls for gender, race, age, and the student's COMPASS English and Math scores to examine differences between the groups.
Findings
Education and skills gain
- The study found that students in the comparison group were significantly more likely to complete the program than students the Welding treatment group. However, there were no students in the treatment group who completed the program for Welding because the program was not fully running for a period of time during the study.
- The study found that students in the comparison group were significantly more like to complete the program than students in the Mechanical Engineering Technology Manufacturing and Design treatment group, but the authors note that the treatment group had no completers, which may have affected the results.
Considerations for Interpreting the Findings
The authors used propensity score matching to create a comparison group; however, they did not account for preexisting differences between the groups in financial disadvantage. These preexisting differences between the groups—and not the intervention—could explain the observed differences in outcomes. In addition, the authors used a cohort from previous enrollment years as a comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the intervention. As stated in the findings section, the treatment programs had no completers, which may also have affected the results. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention.This means we are not confident that the estimated effects are attributable to the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program; other factors are likely to have contributed.