Absence of Conflict of Interest
As the National Evaluation Coordinator for the Workforce Innovation Fund, Abt Associates provided technical advisory support on this study. Therefore, the review of this study was conducted by independent consultants trained in applying the CLEAR causal evidence guidelines.
Citation
Highlights
- The study’s objective was to examine the impact of the New Orleans Office of Workforce Development (OWD) Career Pathways training program on employment, earnings, and job satisfaction.
- This study was a randomized controlled trial that used data from OWD, the Louisiana Workforce Commission, and surveys to compare outcomes of jobseekers who participated in the Career Pathways training program to a control group of jobseekers who did not.
- The study found no statistically significant relationships between the Career Pathways training program and employment, earnings, and job satisfaction. The study found positive statistically significant relationships between the Career Pathways training program and quarterly earnings for individuals who were unemployed at baseline, who were younger than 35 years at baseline, and who were randomized in later cohorts of the program (2017 and later).
- This study receives a low evidence rating because some individuals were included in both the treatment and control groups, introducing a confound. This means we are not confident that any estimated effects are attributable to the Career Pathways training program; other factors are likely to have contributed.
Intervention Examined
Career Pathways
Features of the Intervention
Career Pathways was launched by the New Orleans Office of Workforce Development with a grant from the U.S. Department of Labor through its Workforce Innovation Fund (WIF). Career Pathways served New Orleans, Louisiana residents who were unemployed, underemployed, discouraged (open to work but not actively looking), or employed but seeking a career change or increase in salary. To be eligible, individuals had to complete a mandatory orientation, drug test, test of adult basic skills, and structured interview. Eligibility was determined by meeting literacy and numeracy requirements and demonstrating strong potential for completing the program. Participants were recruited from one-stop centers and community partners in New Orleans.
Career Pathways provided occupation-specific training in advanced manufacturing (electrical, welding, and pipefitting), healthcare (medical coding and billing, pharmacy technician), and information technology. Participants were offered two rounds of training. The first round typically lasted two months, with classes generally held four hours a day, five days a week. Participants who completed the first round had the option to participate in an additional two months of more in-depth training. Training providers included a community college, a computer learning center, and a health system. Participants were provided $6,000 of credit to cover training-related costs, and the program aimed to coordinate with hiring firms to direct new trainees to potential employers.
Features of the Study
This study was a randomized controlled trial. Randomization occurred at the individual level, separately for each training cohort. Within each cohort, individuals who consented to the study were randomized to the treatment group, which was offered the Career Pathways training program, or the control group, which could not enroll in Career Pathways but could access existing services available from one-stop centers and community partners. Some individuals randomized to the control group were randomized to the treatment group in later training cohorts. The analytic sample was 509 individuals: 256 were randomized to the treatment group only, 211 to the control group only, and 42 randomized to the control group and then later randomized to the treatment group in a subsequent cohort. Over half of participants were male, over half were at least 35 years old, and slightly more than half were employed at baseline.
The primary data sources were baseline surveys completed at orientation, program records on training attendance and completion, state administrative employment and earnings records, and telephone surveys inquiring about job satisfaction post-training. The state administrative records provided information on participants’ employment and earnings from the third quarter of 2014 through the third quarter of 2018. The telephone survey was conducted between March 2018 and March 2019.
The authors used a statistical model to compare the outcomes of treatment and control group members. The analysis controlled for average quarterly employment status or earnings prior to randomization, baseline demographics like gender and age, and fixed effects for each cohort, year, and quarter.
Findings
Employment
- The study found no statistically significant relationship between the Career Pathways training program and employment outcomes.
Attitudes
- The study found no statistically significant relationship between the Career Pathways training program and job satisfaction outcomes.
Earnings and Wages
- The study found no statistically significant relationship between the Career Pathways training program and overall quarterly earnings, or quarterly earnings conditional on employment.
- The study found a positive statistically significant relationship between the Career Pathways training program and quarterly earnings for individuals who were unemployed at baseline, who were younger than 35 years at baseline, and who were randomized in later cohorts of the program (2017 and later). The authors consider the 2017 and later cohorts to have received full implementation of the program.
Considerations for Interpreting the Findings
The analytic sample included multiple observations for individuals randomized in subsequent training cohorts. Some individuals had repeated measures of outcomes in both the treatment and control group. These individuals were initially assigned to the control group and then randomized to the treatment group in a later cohort. This presents a confound because observations within each group are non-independent; that is, observations within each group are related to, or influenced by, other observations within the group. Differences in outcomes could be due to individual varying factors within each group and not the Career Pathways program.
Causal Evidence Rating
This study receives a low evidence rating because some individuals were included in both the treatment and control groups, introducing a confound. This means we are not confident that any estimated effects are attributable to Career Pathways and not to other factors.