Skip to main content

Quasi-experimental impact study of NFWS/SIF Workforce Partnership Programs evidence on the effectiveness of Workforce Partnership Programs in Ohio and Wisconsin (Michaelides et al. 2016)

Review Guidelines

Absence of conflict of interest.

Citation

Michaelides, M., Mueser, P., Davis, S., & Mbwana, K. (2016). Quasi-Experimental Impact Study of NFWS/SIF Workforce Partnership Programs Evidence on the Effectiveness of Workforce Partnership Programs in Ohio and Wisconsin. Columbia, MD: IMPAQ International. [Manufacturing Pathways]

Highlights

  • The study's objective was to examine the impact of workforce partnership programs on employment and earnings outcomes. This profile focuses on the evaluation of the Wisconsin Regional Training Partnership (WRTP) Manufacturing Pathways. The authors investigated similar research questions for other sites, the profiles of which can be found here.
  • The study used a nonexperimental design to compare outcomes between individuals receiving services from the Manufacturing Pathways to a matched comparison group receiving state employment services. Using program and unemployment insurance wage data, the authors conducted statistical models to examine differences in outcomes between the groups.
  • The study found that individuals receiving services from the Manufacturing Pathways program were significantly more likely to be employed and earn more than the individuals in the comparison group.
  • This study receives a moderate evidence rating. This means we are somewhat confident that the estimated effects are attributable to WRTP Manufacturing Pathways, but other factors might also have contributed.

Intervention Examined

Wisconsin Regional Training Partnership (WRTP) Manufacturing Pathways

Features of the Intervention

The National Fund for Workforce Solutions/Social Innovation fund (NFWS/SIF) is a collaboration of national foundations that invests in regional funding collaboratives to promote employment and career advancements for low-income individuals and to ensure that employers can obtain a skilled workforce. The Wisconsin Regional Training Partnership (WRTP) was formed in 1992 and funded by NFWS/SIF and other public and private sources. The goal of WRTP was to build a relationship between employers, unions, and workers to enhance the ability of employers in the construction and manufacturing sectors to recruit and develop a diverse and qualified workforce. WRTP collaborated with regional partners including employers, unions, government organizations, community organizations, and training providers. WRTP Manufacturing Pathways offered a variety of trainings and services to unemployed workers interested in manufacturing careers bases on an individual needs assessment. The program offered pre-apprenticeship training, occupational training and certifications (e.g., machinists, operating engineers, sheet metal workers, steamfitters, and welders), career advancement training, job search services, and referrals to partner employers.

Features of the Study

The study used a nonexperimental design to compare the outcomes of individuals receiving services from the Manufacturing Pathways (treatment group) to a sample of individuals who received state employment services (comparison group). The authors formed a matched comparison group that was similar to the treatment group on individual characteristics and employment histories. The treatment group included 88 unemployed individuals and the comparison group included 19,844 individuals. The majority of program participants were men (84 percent) and had at least a high school diploma (79 percent). Fifty-seven percent of participants were Black and 31 percent were White; more than half were over the age of 35 (60 percent). Data sources included WRTP Manufacturing Pathways program data and employment service data that provided participant socioeconomic and demographic information upon program entry and the type of services that participants received during the study, as well as state unemployment insurance wage data that provided quarterly earnings for all participants. The authors used a statistical model to compare employment and earnings outcomes between the groups through a six-quarter follow-up period.

Findings

Earnings and wages

  • The study found that treatment participants had significantly higher earnings than comparison participants across six quarters.

Employment

  • The study found that treatment participants were significantly more likely to be employed than comparison participants for all quarters, except quarter six where the difference between the groups was not significant.

Considerations for Interpreting the Findings

The study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.

Causal Evidence Rating

The quality of causal evidence presented in this report is moderate because it was based on a well implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to WRTP Manufacturing Pathways, but other factors might also have contributed.

Additional Sources

Michaelides, M., Poe-Yamagata, E., & Mbwana K. (2013). Outcomes Assessment Study of NFWS/SIF-funded Programs. Boston, MA: National Fund for Workforce.

Reviewed by CLEAR

August 2022

Topic Area