Citation
Vinokur, A., van Ryn, M., Gramlich, E., & Price, R. (1991). Long-term follow-up and benefit-cost analysis of the Jobs Program. Journal of Applied Psychology, 76(2), 213-219.
Highlights
-
- The study’s objective was to examine the impact of the Jobs Program on long-term earnings and monthly hours worked.
- The authors conducted a randomized controlled trial (RCT). Researchers administered a pre-test two weeks before the intervention and three post-tests at 1, 4, and 30 months after the intervention comparing the outcomes of Jobs Program participants against a control group.
- The study found no statistically significant relationships between the Jobs Program and increased long-term earnings or time in work during the follow-up period.
- The quality of causal evidence presented in this report is low because the study was an RCT with post-intervention subsampling based on participation in the intervention. In addition, the authors did not include sufficient controls in their analysis. This means we are not confident that any estimated effects would be attributable to the Jobs Program; however, the study found no statistically significant effects.
Intervention Examined
The Jobs Program
Features of the Intervention
The Jobs Program consisted of eight three-hour sessions conducted on four mornings of each of two weeks. Sessions were led by male-female pairs of trainers who had received about 80 hours of formal training and were regularly observed by members of the research team. Sessions focused on (1) establishing trust in the trainers, (2) building participants’ self-esteem and motivation, and (3) teaching specific job search skills. Skills targeted included identifying and effectively expressing competencies, using social connections to find job leads, contacting potential employers, preparing applications, and interviewing.
In addition to teaching job search skills, the intervention focused on the mental and emotional well-being of participants. Session time was dedicated to “psychologically inoculating” participants against losing their motivation in the face of prolonged unemployment by teaching problem-solving skills designed to help participants anticipate setbacks and adapt when they arose. Trainers also offered direct encouragement and support during the intervention and provided opportunities for other participants to support one another.
Features of the Study
The authors conducted an RCT, randomly assigning 1,122 people to either treatment or control groups. Those in the treatment group were given the opportunity to attend the Jobs Program, whereas those in the control group were provided with self-instructional materials on general job search strategies. Study participants were recruited from four unemployment compensation offices across southern Michigan. The recruitment process limited the study to people who were applying for or receiving unemployment compensation in the state. Individuals were considered ineligible to participate if they were within two years of retirement, expected to be recalled to their previous job, were judged by experimenters as showing clear indications of mental illness, or reported being unemployed for more than four months. Researchers administered a pre-test two weeks before the intervention began and two post-tests one and four months after the intervention ended. Impacts were estimated by comparing unadjusted treatment and control group means.
Findings
Earnings and wages
- The study found no statistically significant relationships between the Jobs Program and long-term earnings or time in work during the follow-up period.
Considerations for Interpreting the Findings
Attrition is difficult to discern in this study because the authors did not provide sample sizes by treatment condition at each follow-up and presented different sample sizes in different studies describing the same experiment. However, the fact that only a subset of participants who were assigned to the Jobs Program but dropped out were sampled for the follow-up surveys means that random assignment was compromised and the study cannot receive a high causal evidence rating. This is because the decision to participate in Jobs occurred after random assignment; to preserve the integrity of the random assignment process, the authors would have had to attempt to survey all of those randomly assigned to the Jobs Program, including all those who did not ultimately participate in it.
Compromised RCTs can receive a moderate causal evidence rating if they include controls for demographic and pre-intervention variables specified in CLEAR’s topic area review protocol. Because the study did not include all the required controls in the analysis, it cannot receive a moderate causal evidence rating. In particular, the study did not include controls for employment stability, pre-intervention employment, or earnings measured more than one year before random assignment.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the study was an RCT with post-intervention subsampling based on participation in the intervention. In addition, the authors did not include sufficient controls in their analysis. This means we are not confident that any estimated effects would be attributable to the Jobs Program; however, the study found no statistically significant effects.
Additional Sources
Caplan, R., Vinokur, A., Price, R., & van Ryn, M. (1989). Job seeking, reemployment, and mental health: A randomized field experiment in coping with job loss. Journal of Applied Psychology, 74(5), 759-769.
Vinokur, A., Price, R., Caplan, R., van Ryn, M., & Curran, J. (1995). The JOBS I preventive intervention for unemployed individuals: Short- and long-term effects on reemployment and mental health. In L. Murphy, J. Hurrell, S. Sauter, & G. Keita (Eds.), Job stress interventions (pp. 125-138). Washington, DC: American Psychological Association (NIOSH).