Absence of conflict of interest.
Citation
Singer, S. (2018). Get IT project evaluation: Final report. Syracuse, NY: Hezel Associates.
Highlights
- The study’s objective was to examine the impact of the enhanced Growing the Economy through Information Technology (Get IT) curricula on student education outcomes.
- The study used a nonexperimental design to compare the outcomes of students in enhanced Get IT courses to a comparison group of students not taking enhanced IT courses. Using administrative records, the author conducted statistical tests to examine the differences between groups.
- The study found that students in enhanced Get IT courses were significantly more likely to remain in school than students who were not in enhanced Get IT courses.
- The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to NHTI’s enhanced Get IT program; other factors are likely to have contributed.
Intervention Examined
NHTI’s Get IT Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Upon receipt of a TAACCCT grant in 2015, NHTI expanded and improved some of their Growing the Economy through Information Technology (Get IT) courses. The improved courses included: 1) the development of common core IT courses that can be stacked and transferred across the Community College System of New Hampshire; 2) new IT-related certificate programs that are relevant to employers; 3) the implementation of programs such as career counseling, latticed programming, and improved prior learning assessment (PLA) procedures to facilitate faster IT course completion; and 4) the development of new Digital Fabrication Lab that combines IT and industrial design. The program was designed to target Trade Adjustment Assistance-eligible workers and other adults. Additional services made available to students through the newly developed program included a dedicated career specialist to assist students with job and internship placement, accelerated courses, the integration of a soft skill "Mindful Communication" program, and alternate avenues for students to earn credits for previous experiences. Aspects of some Get IT courses were not enhanced using grant funds (“non-TAACCCT”).
Features of the Study
The author used a nonexperimental design to assess retention and certificate or degree attainment at NHTI Community College in Concord, New Hampshire. When measuring retention, 716 students in TAACCCT programs were compared to 4,498 students in non-TAACCCT programs. All students first enrolled in the Fall 2015 to Spring 2018 semesters at NHTI. When measuring credential attainment, 293 students TAACCCT programs were compared to 1,979 students in non-TAACCCT programs. Certificate students included those who first enrolled in the Spring 2017 semester or earlier, while degree attainment students included those who first enrolled in Spring 2016 or earlier. Using data from NHTI administrative records, the author compared retention and certificate or degree attainment between the treatment and comparison groups and conducted statistical tests on retention outcomes.
Findings
Education and skills gain
- The study found that TAACCCT Get IT participation was significantly associated with a higher likelihood of remaining in the program one year after entering than students in non-TAACCCT courses (1.4 times more likely).
- The study found that TAACCCT Get IT participation was associated with a higher percentage of Associate degree or certificate awards (31%) than comparison group participants (23%). Tests of significance were not reported.
Considerations for Interpreting the Findings
The author did not provide results of the regression model that controlled for TAACCCT status, age, and GPA, and did not account for other factors that could have affected the difference between the treatment and comparison groups, such as race/ethnicity, gender, a pre-intervention measure of degree of financial disadvantage, and a pre-intervention measure of education/training. These preexisting differences between the groups—and not the Get IT program—could explain the observed differences in outcomes. Also, the author did not provide information on the types of courses that were enhanced using TAACCCT funds and those that were not enhanced. Thus, the treatment and comparison courses may differ in systematic and unobserved ways, affecting the observed outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to the NHTI-enhanced Get IT program; other factors are likely to have contributed.