Absence of conflict of interest.
Citation
Fogg, N. P., Harrington, P. E., & Khatiwada, I. (2016). Impact evaluation of the New England Institute of Technology's Shipbuilding/Marine Advanced Manufacturing Institute (SAMI) Program TAACCCT II grant. Philadelphia, PA: Center for Labor Markets and Policy, Drexel University.
Highlights
- The study’s objective was to examine the impact of the Shipbuilding/Marine Advanced Manufacturing Institute (SAMI) program on earnings and employment outcomes.
- Using data from the SAMI program student database and unemployment insurance data from the Rhode Island Department of Labor and Training, the authors conducted a nonexperimental study to compare the outcomes of SAMI participants to a matched comparison group.
- The study found that participating in the SAMI program was significantly associated with higher rates of employment and higher earnings.
- The quality of casual evidence provided in this study is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the SAMI program; other factors are likely to have contributed.
Intervention Examined
The Shipbuilding/Marine Advanced Manufacturing Institute (SAMI)
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Shipbuilding/Marine Advanced Manufacturing Institute (SAMI) program was created at the New England Institute of Technology using TAACCCT grant funds. SAMI allowed the college to enhance classroom education, provide opportunities for hands-on training, improve relationships with local employers, upgrade their physical infrastructure, and offer new certifications. SAMI focused on two programs: welding and machine trades. Many SAMI students were referred by career centers affiliated with the Rhode Island Department of Labor and Training; most had been laid off and had experienced long-term unemployment. The study authors note there was a screening program, but do not describe this process or the eligibility requirements for program admission.
Features of the Study
The study used a nonexperimental design to compare students who participated in the SAMI program to non-participants. The authors matched SAMI participants to similar nonparticipants using propensity scores developed from demographic, education, and occupation information. The treatment group included 290 students who participated in the SAMI program and were enrolled between 2014 and 2015. The comparison group consisted of 298 individuals who had an active unemployment insurance (UI) claim in 2014 or 2015. Using data from the SAMI program student database and UI data from the Rhode Island Department of Labor and Training, the authors examined differences in outcomes between the treatment and comparison groups. Outcomes included median income, employment status, and percent of potential quarters employed (measured as the actual quarters employed relative to the potential quarters of employment).
Findings
Earnings and Wages
- The study found a significant relationship between SAMI program participation and earnings in the second quarter of 2016 (which was one quarter after the program ended). SAMI participants earned a median quarterly wage of $7,713 compared to a median quarterly wage of $6,500 for individuals in the comparison group during the same quarter.
Employment
- The study found a significant relationship between SAMI program participation and employment the second quarter of 2016, with a higher proportion of SAMI participants (77%) employed than individuals in the comparison group (66%).
- The study also found that program participation was significantly associated with a higher percent of potential quarters employed, with SAMI participants employed for 79% of the eligible quarters while comparison group individuals were employed for 73% of eligible quarters.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating individuals to compare to students who were enrolled in the SAMI program. However, the authors did not appropriately control for other factors that could have affected the difference between the treatment and comparison groups, such as a pre-intervention degree of financial disadvantage, employment, or earnings. These preexisting differences between the groups—and not the SAMI program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the SAMI program; other factors are likely to have contributed.