Absence of conflict of interest.
Citation
The New Growth Group, LLC & The Ohio Education Research Center at The Ohio State University. (2018). The Ohio Technical Skills Innovation Network (Ohio TechNet) Trade Adjustment Assistance Community College Career Training Grant (TAACCCT) Final Evaluation Report. Retrieved from https://oerc.osu.edu/sites/oerc/themes/oerc/publications/LCCC%20OTN%20Lead%20Team%20Evaluation%20Final%20Report.pdf
Highlights
- The study’s objective was to evaluate the impact of advanced manufacturing programs offered through the Ohio Technical Skills Innovation Network (OTN) initiative on education, earnings, and employment outcomes.
- The study used a nonexperimental design to compare outcomes of students enrolled in OTN-affected programs/core courses to a matched comparison group.
- The study found that OTN participation was associated with a significant increase in program completion, program retention, credit hour completion, continuing further education, credential attainment, immediate employment, and job retention. Program participation was also associated with a significant decrease in retention in other programs and earning a degree.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the OTN grant programs; other factors are likely to have contributed.
Intervention Examined
The Ohio Technical Skills Innovation Network Initiative
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2014, the Ohio Technical Skills Innovation Network (OTN) was awarded a TAACCCT grant to address local workforce challenges in advanced manufacturing. The OTN is comprised of 11 community colleges in Ohio. With the grant, the OTN colleges enhanced and scaled programming that offered accelerated education and skills training for adults while meeting industry labor demands. Specifically, the curriculum was aligned with business needs, new technology was leveraged in instruction delivery, industry credentials were integrated into credit pathways, online hybrid courses were developed, more intensive student support services and advising were provided, and there was more active engagement with local employers. The five certificate and degree pathways enhanced by the grant included welding, Computer Numerical Control (CNC)/machining, industrial maintenance, digital fabrication/industrial automation, and occupational safety.
Features of the Study
The study used a nonexperimental design to determine the impact of OTN grant-funded programming on education, earnings, and employment outcomes. The treatment group included students enrolled in a program or core course enhanced by the grant. The comparison group included students enrolled in programs and courses similar to those enhanced by the grant at non-OTN academic institutions in Ohio. The authors matched OTN participants to similar nonparticipants using propensity scores developed from demographic information. Across the 11 colleges, the sample included 1,471 in the treatment group and 1,470 in the comparison group. Data sources included OTN program data, Higher Education Information (HEI), and the Ohio Longitudinal Data Archive (OLDA). The authors used statistical models to examine differences in outcomes between the groups. Outcomes included program completion in terms of earning a degree or certificate, program retention, retention in another program, continuing further education, credential and degree attainment, credit hour accumulation, employment one quarter after program exit, employment for three consecutive quarters after program exit, and earnings.
Study Sites
- Cincinnati State and Technical Community College in Cincinnati, Ohio • Columbus State Community College in Columbus, Ohio
- Cuyahoga Community College in Cleveland, Ohio
- Eastern Gateway Community College in Steubenville, Ohio
- Lakeland Community College in Kirtland, Ohio
- Lorain County Community College in Elyria, Ohio
- Owens Community College in Perrysburg, Ohio
- Rhodes State College in Lima, Ohio
- Sinclair Community College in Dayton, Ohio
- Stark State College in Canton, Ohio
- Zane State College in Zanesville, Ohio
Findings
Education and skills gain
- Relative to the comparison group, the study found that OTN participation was associated with a significant increase in program retention (53% vs. 41%), program completion/earning any credential (24% vs. 16%), earning less than a one-year credential (64% vs. 24%), and continuing education at a different college (31% vs. 16%).
- However, the study found that comparison students were significantly more likely to be retained in another program (10%) and earn a degree (78%) than treatment students (1% and 38%, respectively).
- For those enrolled in for-credit programming, OTN participation was significantly associated with higher likelihoods of completing any credit hours (92% vs. 78%).
- The study did not find a significant relationship between OTN participation and earning 1-to-2-year certificates or the mean number of credit hours completed.
Earnings and wages
- The study found no significant relationship between OTN participation and earnings increase post-program enrollment.
Employment
- The study found that for unemployed individuals, OTN participation was significantly associated with higher rates of employment one quarter (52% vs. 26%) and three quarters after program completion (46% vs. 18%).
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to OTN participants. However, the authors did not appropriately control for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage and race/ethnicity. These preexisting differences between the groups—and not the OTN grant programs—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the OTN grant programs; other factors are likely to have contributed.