Absence of conflict of interest.
Citation
de la Mora, A., Abraham, W., & Callen, E. (2018). Iowa’s Information Technology, Healthcare, Utilities, and Manufacturing Network Statewide Consortium (IHUM) final evaluation report. Ames, IA: Iowa State University, Research Institute for Studies in Education.
Highlights
- The study’s objective was to evaluate the effects of the Iowa’s Information Technology, Healthcare, Utilities, and Manufacturing (IHUM) Network Consortium project on education outcomes.
- The authors used a nonexperimental design to compare the education outcomes of the IHUM treatment group to a comparison group comprised of both historical and concurrent cohorts of students.
- The study found that IHUM participation was significantly related to higher rates of program diplomas earned and lower rates of earning an Associate’s degree.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention and did not include sufficient controls. This means we are not confident that the estimated effects are attributable to IHUM; other factors are likely to have contributed.
Intervention Examined
Iowa’s Information Technology, Healthcare, Utilities, and Manufacturing (IHUM) Network Consortium
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Iowa’s Information Technology, Healthcare, Utilities, and Manufacturing (IHUM) Network Consortium project was formed in response to a shortage of skilled workers in information technology, healthcare, utilities, and manufacturing. The main objective of the IHUM project was to provide support to participating community colleges to develop and expand the number of programs and types of credentials. The IHUM project offered over 105 programs of study within four sectors: information technology, healthcare, utilities, and manufacturing. The project offered several services to assist students, which included career navigation, third party certification, and credits for prior learning. The project also sought to increase the visibility of the programs of study through statewide and regional marketing efforts to increase student participation in the various programs.
Features of the Study
The nonexperimental study took place across fifteen college campuses in Iowa that implemented the IHUM project. The authors compared the outcomes of students who participated in the IHUM program to those who did not. The treatment group included three cohorts of students enrolled in IHUM programs during fall 2015 and 2016 and spring 2016. The comparison group included five cohorts of students: two historic and three concurrent. The historic cohorts included students from the same community colleges that implemented the IHUM programs and enrolled in the fall 2013 and 2014 semesters prior to the implementation of the IHUM program. The concurrent cohorts included students enrolled in the same programs of study at non-IHUM colleges during the grant period. Study participants included 3,367 students in the treatment group and 3,563 students in the comparison group. Using educational data from the Iowa Department of Education (IDoE), the authors examined differences in credential completion (Associate’s degree, program certificate, or program diploma) between the groups.
Findings
Education and skills gain
- The study found a significant relationship between the IHUM program and earning a program diploma, with higher proportions of IHUM students earning a program diploma (19%) than non-IHUM students (8%).
- However, the study also found that IHUM participation was significantly associated with lower rates of Associate degree completion, with lower proportions of IHUM students earning an Associate’s degree (13%) than non-IHUM students (25%).
- The study did not find a significant relationship between the IHUM program and earning a program certificate.
Considerations for Interpreting the Findings
The study authors used several treatment and comparison groups across multiple cohorts. Comparison groups consisted of both historical cohorts from years prior to the implementation of the IHUM program as well as concurrent cohorts from different colleges not employing the IHUM program. These program and time-varying differences between the participants can lead to a confound. Additionally, the authors did not ensure that the treatment and comparison groups were similar prior to the intervention or adequately control for key differences between the groups. The composition of the treatment and comparison groups significantly varied by gender, race/ethnicity, and completion of developmental math courses. These preexisting differences between the groups—and not the IHUM program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention and did not include sufficient controls. This means we are not confident that the estimated effects are attributable to IHUM; other factors are likely to have contributed.