Absence of conflict of interest.
Citation
Farrell, R., Harris, D., Meyers, H., & Ratmeyer, S. (2017). STEM-Connect at the University of Vermont, College of Engineering and Mathematical Sciences: Program evaluation final report. Swanton, VT: Core Research and Evaluation.
Highlights
- The study’s objective was to examine the effects of the University of Vermont’s (UVM) Science, Technology, Engineering and Math (STEM) Connect program on earnings and employment outcomes.
- The study used a nonexperimental design to compare the outcomes of students who were enrolled in the UVM STEM-Connect program to a matched comparison group of students who took a STEM-Connect course but were not enrolled in the STEM-Connect program.
- The study found no statistically significant relationships between STEM-Connect program participation and average quarterly wages or employment rates.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the UVM STEM-Connect program; other factors are likely to have contributed.
Intervention Examined
The UVM STEM-Connect Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The University of Vermont’s (UVM) College of Engineering and Mathematical Sciences department received a TAACCCT grant to design the STEM-Connect program with the goal of bolstering UVM’s ability to prepare students for the changing technology field and for employment in technology industries. The program was designed to target Trade Adjustment Assistance (TAA) eligible workers, veterans, dislocated workers, underemployed or unemployed adults, and those new to the STEM fields of study. Students in STEM-Connect certificate programs were able to earn credentials in Computer Software Certificates in Software Development, Web Development, Cybersecurity, Master’s Preparation and Self Design; Computer-Aided Engineering Technology; Complex Systems (master’s level); and Pre-Actuarial (Actuarial Science). STEM-Connect is administered in a traditional classroom setting as well as online, and includes mentoring and internship coordination. In addition to standard support services such as access to UVM's financial aid offices, counseling, and career development, STEM-Connect provides non-traditional students with enhanced recruitment, counseling, tutoring and academic advising opportunities.
Features of the Study
The nonexperimental study took place at the UVM’s College of Engineering and Mathematical Sciences department in Burlington, Vermont. The treatment group consisted of 331 UVM students who participated in the TAACCCT-funded STEM-Connect program from 2014 to 2016, while the comparison group consisted of 2,088 UVM students who took a STEM-Connect course as part of their regular university program but did not participate in STEM-Connect. The authors matched STEM-Connect program participants to similar nonparticipants using demographic characteristics. Using data from UVM's student database and the Vermont Department of Labor (VDOL) database, the authors conducted chi-square analyses to compare outcomes between the treatment and comparison groups. Outcomes included average quarterly wages and employment rates.
Findings
Earnings and wages
- The study did not find a statistically significant relationship between STEM-Connect program participation and average quarterly wages.
Employment
- The study did not find a statistically significant relationship between STEM-Connect program participation and employment rates.
Considerations for Interpreting the Findings
The authors did not account for preexisting differences between the groups before program participation. Specifically, they did not account for differences in race/ethnicity when creating the matched comparison group which is required by the review protocol. These preexisting differences between the groups—and not the STEM-Connect program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the UVM STEM-Connect program; other factors are likely to have contributed.