Absence of conflict of interest.
Citation
Dockery, J., Bottomley, M., Murray, C., Tichnell, T., Stover, S., Schroeder, N., Hance, D., Fruchey, K., & Franco, S. (2018). Northwest State Community College of Ohio Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) Initiative (TAACCCT Grant TC-26481-14-60-A-39). Dayton, OH: Wright State University, Applied Policy Research Institute.
Highlights
- The study’s objective was to examine the impact of the Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) initiative on earnings and employment outcomes.
- The authors used a nonexperimental design to compare employment and earnings outcomes of students enrolled in the IAM iSTAR program to a matched cohort of students from other local community colleges enrolled in the same or similar programs of study.
- The study found that IAM iSTAR program participation was significantly associated with a higher likelihood of finding and retaining employment for non-incumbent workers.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the IAM iSTAR program; other factors are likely to have contributed
Intervention Examined
Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) Initiative
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Northwest State Community College (NSCC) in Ohio used TAACCCT funding to implement the Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) initiative. The grant funds were awarded to the college to assist in the conversion of traditional industrial technology courses to an intensive modularized hybrid format. The industrial technologies division at NSCC trains industrial automation maintenance technicians in the advanced manufacturing industry. The IAM iSTAR program included several components and strategies which included: developing new and modified competency-based curriculum, transforming the academic process to accelerate student learning and completion (e.g., virtual trainers), creating innovative instructional media and technology to accelerate learning (e.g., online classes and open labs), and redesigning student support services to increase student success (e.g., career coaching).
Features of the Study
The study used a nonexperimental design to compare the employment and earnings outcomes of students who were enrolled in the IAM iSTAR program to students who were not in the program during the same time frame. The authors matched IAM iSTAR participants to similar nonparticipants using propensity scores developed from demographic and academic information. The treatment group consisted of 589 NSCC students who were enrolled in the IAM iSTAR program. The comparison group consisted of 233 students from other colleges serving rural areas in Ohio that had no current or previous TAACCCT grant awards. The comparison students were enrolled during the same time period as the treatment group and in similar programs of study. Using administrative unemployment insurance data from the state of Ohio, the authors conducted statistical analyses to examine differences between the groups in gaining employment, retaining employment, and wage increases.
Findings
Earnings and wages
- The study did not find a significant relationship between participation in the IAM iSTAR program and wage increases.
Employment
- The study found a significant relationship between the IAM iSTAR program and gaining employment for non-incumbent workers, with treatment students being 4.5 times more likely to find employment within one quarter of exiting the program relative to students in the comparison group.
- The study also found a significant association between the IAM iSTAR program and retaining employment, with a higher likelihood of treatment students being employed for two consecutive fiscal quarters after program completion than students in the comparison group.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to students who were enrolled in the IAM iSTAR program. However, the authors did not appropriately control for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not the IAM iSTAR program—could explain the observed differences in outcomes. Additionally, the authors used a concurrent cohort of students that attended different colleges to create the comparison group. The college-varying characteristics of the comparison group created a confounding factor. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the IAM iSTAR program; other factors are likely to have contributed.