Absence of conflict of interest.
Citation
Haviland, S., Van Noy, M., Kuang, L., Vinton, J., & Pardalis, N. (2018). Evaluation of Richland College's Veterans-Focused Engineering Technology Project: Final Report. New Brunswick, NJ: Rutgers University Education and Employment Research Center.
Highlights
- The study’s objective was to examine the impact of Richland College’s Veterans-Focused Engineering Technology Project on education outcomes.
- The authors used a nonexperimental design to compare education outcomes of program participants to a matched comparison group one and two years after enrollment.
- The study found that program participation was significantly associated with fewer credit hours earned one year after enrollment.
- The quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the Veterans-Focused Engineering Technology Project; other factors are likely to have contributed program.
Intervention Examined
The Veterans-Focused Engineering Technology Project
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Veterans-Focused Engineering Technology Project was a TAACCCT-funded initiative to improve the Manufacturing and Engineering Technology (MET) programs at Richland College, which is located outside of Dallas, Texas. The program began in Fall 2015, and included curriculum design/redesign, expansion of credentials to the MET programs, student support services (e.g., advising, career services), and improvements to existing laboratory facilities. Recruitment targeted veterans; however, non-veterans participated in the program. Recruitment initiatives included providing information about the program at college fairs, job fairs, and community events.
Features of the Study
The study used a nonexperimental design to compare the outcomes of first-time students who participated in Richland’s TAACCCT Veterans-Focused Engineering Technology Project program to first-time students who did not participate. The treatment group included students who enrolled in TAACCCT programs between Fall 2015 and Spring 2018. The comparison group included students enrolled in the Computer Information Technology (CIT) program between Fall 2015 and Summer 2017. The authors matched Richland’s TAACCCT program participants to similar nonparticipants using propensity scores developed from demographic and education information. Study participants included 114 students in the treatment group and 294 students in the comparison group. Using administrative data from Richland College, the authors conducted statistical models to examine differences in outcomes between the groups. Outcomes included credit hour completion (one and two years after enrollment), immediate retention (continued enrollment to the next term) and long-term retention (within the two years following initial enrollment).
Findings
Education and skills gain
- The study found that program participation was significantly related to lower credit completion one year after enrollment, with students in the treatment group earning an average of 3 credits less than students in the comparison group.
- However, the study did not find a significant relationship between program participation and credit hour completion two years after enrollment.
- The study did not find significant relationships between program participation and immediate retention or long-term retention.
Considerations for Interpreting the Findings
Although the authors accounted for baseline characteristics, they did not control for baseline education outcomes in their analytic model as required by the review protocol. The preexisting differences between the groups on this variable—and not Richland’s Veterans-Focused Engineering Technology Project—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the Veterans-Focused Engineering Technology Project, other factors are likely to have contributed.