Absence of conflict of interest.
Citation
WorkED Consulting. (2017). North Dakota State College of Science TAACCCT Project: North Dakota Advanced Manufacturing Skills Training Initiative (NDAMSTI) final evaluation report. Burke, VA: WorkED Consulting.
Highlights
- The study’s objective was to examine the impact of the North Dakota Advanced Manufacturing Skills Training Initiative (NDAMSTI) on education, earnings, and employment outcomes.
- The authors used a nonexperimental design to compare the outcomes of the NDAMSTI program participants to a historic cohort of students enrolled in the same welding program five years prior to the grant funding.
- The study found that NDAMSTI participation was positively related to program completion and negatively related to rates of continuing education, employment, and wage increases. However, the study did not include tests of statistical significance.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the NDAMSTI program; other factors are likely to have contributed.
Intervention Examined
The North Dakota Advanced Manufacturing Skills Training Initiative (NDAMSTI)
Features of the Intervention
The U.S. Department of Labor’s (DOL) Trade Adjustment Assistance for Community Colleges and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
North Dakota State College of Science (NDSCS) was awarded a TAACCCT grant to establish the North Dakota Advanced Manufacturing Skills Training Initiative (NDAMSTI). The main goal of the NDAMSTI program was to upgrade skills for trade impacted workers, other dislocated workers, and veterans to pursue jobs and careers in the high-demanding fields of welding, manufacturing, and mechatronics. Features of the program included developing and expanding online learning options, developing new curriculum tailored to adult learners, building stackable programs and credentials, expanding hours and locations for classes and access to programming, promoting degree completion options, and enhancing transfer and articulation to four-year universities.
Features of the Study
The nonexperimental study was conducted at North Dakota State College of Science (NDSCS) in Wahpeton, North Dakota and compared students who participated in the NDAMSTI program to students who did not participate. The authors matched NDAMSTI participants to similar non-participants using demographic information. The treatment group included 300 students who participated in the NDAMSTI program and were enrolled in the NDSCS welding program from Fall 2016 to Spring 2017. The comparison group was a historical cohort composed of 325 students that were enrolled in the same NDSCS welding program from Fall 2009 to Spring 2013, five years prior to the implementation of the NDAMSTI program. Data sources included baseline and endline surveys, NDSCS administrative data, and state employment and education data. The authors examined differences in program completion, continuing education, wages, and employment status between treatment and comparison groups. However, the study did not include tests of statistical significance.
Findings
Education and skills gains
- The study found that NDAMSTI participation was related to program completion, with program participants having higher completion rates than comparison students (81% vs. 65%).
- However, the study found that a higher percentage of students in the comparison group continued their education compared to students in the treatment group (53% vs. 47%).
Earnings and wages
- The study found a negative association between program participation and improved wages, with higher rates of wage increases in the comparison group (84%) than the treatment group (78%).
Employment
- The study found that NDAMSTI participation was related to lower rates of employment, where 64% of treatment participants were employed after program completion compared to 66% of participants in the comparison group.
Considerations for Interpreting the Findings
The authors created a matched group of non-participants to compare to NDAMSTI participants. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention education/training or degree of financial disadvantage as required by the protocol. These preexisting differences between the groups—and not the NDAMSTI program—could explain the observed differences in outcomes. Additionally, the authors used a cohort of NDSCS students from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the economy) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the NDAMSTI program; other factors are likely to have contributed.