Citation
Foley, K., Pallas, D., Forcehimes, A., Houck, J., Bogenschutz, M., Keyser-Marcus, L., & Svikis, D. (2010). Effect of job skills training on employment and job seeking behaviors in an American Indian substance abuse treatment sample. Journal of Vocational Rehabilitation, 33(2010), 181-192.
Highlights
- The study’s objective was to examine the impact of workshops for job seekers on employment and training outcomes.
- The study used a randomized controlled trial (RCT) design, in which eligible participants were randomly assigned to the treatment group, which could participate in the job seeker workshops, or the control group, which could not participate. The authors used follow-up surveys three and six months after random assignment to measure self-reported employment and training outcomes.
- The study found that three months after workshop participation, the treatment group spent fewer hours on training activities than the control group; the study did not find any significant impacts of the job seekers’ workshop on the number of hours spent on work or percentage of each group who were employed at the three-month follow-up.
- The quality of causal evidence presented in this report is high for the percentage employed and hours spent on work and training outcomes collected at the three-month follow-up because those outcomes were based on a well-implemented RCT with low attrition. This means we are confident that the estimated effects are attributable to the job seekers’ workshop, and not to other factors. However, the quality of causal evidence presented in this report is low for the days to employment outcomes at the three-month follow-up and all outcomes at the six-month follow-up. This means we are not confident that the estimated effects are attributable to the job seekers’ workshop; other factors are likely to have contributed.
Intervention Examined
Job Seekers’ Workshop
Features of the Intervention
The job seekers’ workshops took place at the Na’Nizhoozhi Center, a substance abuse treatment center in Gallup and McKinley County, New Mexico. The workshops consisted of three four-hour sessions focused on individualized practice and feedback on interview skills. The workshops also covered locating jobs, completing job applications, and writing resumes. Individuals were eligible for the program if they were American Indian adults, were enrolled at the Na’Nizhoozhi Center for at least 10 days, were unemployed or underemployed, and were interested in participating in the labor market.
Features of the Study
The study used an RCT design in which eligible participants were assigned to either the treatment group, which could attend the job seekers’ workshops, or the control group, which could not attend the workshops but instead watched two videos on interview skills. The treatment group consisted of 53 people; the control group had 49 members. Both groups were predominantly male and American Indian. The average age of the treatment group was 35 years old and for the control group it was 38 years old. The authors used three follow-up surveys to measure self-reported employment and training outcomes one, three, and six months after the job seekers’ workshop ended, although the study did not report findings from the one-month follow-up. The authors used a statistical model to compare the outcomes of treatment and control group members, but did not account for preexisting differences between the groups.
Findings
- Employment. The study found no statistically significant relationship between the job seekers’ workshop and employment outcomes at the three-month follow-up.
- Education and training. The study found that those in the job seekers’ workshop group spent significantly fewer hours on training activities, compared with those in the control group, at the three-month follow-up.
Considerations for Interpreting the Findings
The authors noted that one individual in the control group had many more reported training hours than any other participant. The authors tested whether this individual’s extreme results drove the reported impacts of the program. When the authors excluded this individual from the analysis, the authors no longer found statistically significant results, suggesting that this one person might have driven the study’s reported findings on number of hours in training programs at the three-month follow-ups.
Although the study was an RCT, the following outcomes had high attrition: the days to employment outcome in the three-month follow-up and all of the outcomes in the six-month follow-up. The authors did not demonstrate that the treatment and control groups were similar at baseline and did not account for existing differences between the groups being compared in their analysis. Therefore, these outcomes receive a low evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is high for the percentage employed and hours spent on work and training outcomes collected at the three-month follow-up period. These results were based on a well-implemented RCT with low attrition. This means we would be confident that any estimated effects for these outcomes would be attributable to the job seekers’ workshop and not to other factors.
The quality of causal evidence presented in this report is low for days to employment outcomes at the three-month follow-up and all outcomes at the six-month follow-up because there was high attrition and the authors did not account for existing differences between the treatment and control groups. This means we are not confident that the estimated effects are attributable to the job seekers’ workshop; other factors are likely to have contributed.