Citation
Penk, W., Drebing, C. E., Rosenheck, R, A., Krebs, C., Van Ormer, A., & Mueller, L. (2010). Veterans Health Administration transitional work experience vs. job placement in veterans with co-morbid substance use and non-psychotic psychiatric disorders. Psychiatric Rehabilitation Journal, 33(4), 297-307.
Highlights
- The study’s objective was to examine the impact of the Veterans Health Administration (VHA) Transitional Work Experience (TWE) program on employment and income for veterans with substance abuse and non-psychotic psychiatric disorders, compared with job placement (JP) services only.
- The study was a randomized controlled trial carried out at one VHA medical center site over four years. Eligible veterans were randomly assigned to either the treatment group, which could participate in the TWE program, or to the control group, which could not participate in TWE but could receive JP services. The authors used data from participant surveys to measure impacts.
- The study found that members of the treatment group worked significantly more hours in paid employment (including employment sponsored by the TWE program itself) and significantly more total weeks in paid employment, compared with the control group. The study also found that the treatment group earned significantly higher total income for the year (including noncompetitive TWE employment income) than the control group.
- The quality of causal evidence presented in this report is low because the authors did not provide enough information to calculate attrition and did not include sufficient controls in their analysis to establish that treatment and control groups were similar on observable characteristics at baseline for the analytic sample. This means we are not confident that the estimated effects are attributable to TWE. Other factors are likely to have contributed.
Intervention Examined
The Transitional Work Experience (TWE) Program
Features of the Intervention
The TWE program placed participants in structured, paid employment at companies contracted with the program in the community or directly within VHA medical centers. Participants progressed through different job placements, in terms of hours worked, difficulty of the position, and level of independence given in the role. Program participants continued any clinically warranted treatment through the VHA medical centers and received job-coaching services. TWE also assisted participants with identifying goals and completing job searches. Formal assessments and skill-building were not a primary focus of the program. The length of the program was tailored to the needs of the participants and placements were intended to improve professional skills through the experience of paid employment.
To participate in the study, participants had to meet the following inclusion criteria: (1) be eligible for VHA services; (2) meet criteria for psychiatric diagnosis of bipolar disorder, major depression, post-traumatic stress disorder, or other anxiety disorder, and have current or life-time drug or alcohol abuse, with substance use within the past 90 days; (3) not meet criteria for schizophrenia or other schizoaffective disorder diagnosis; (4) not have a recent psychiatric hospitalization or suicidal and homicidal thoughts in the past six months; (5) have worked at least one day of competitive employment in the past three years, have the goal of returning to work and not have medical restrictions that would limit return to work; (6) be younger than 56, with at least 10 years of formal education; and (7) and be likely to maintain contact with research staff.
Features of the Study
From 2006 to 2010, 89 eligible veterans entered TWE services at the Bedford, Massachusetts, VA program. Among them, 50 veterans were randomly assigned to the treatment group, which could participate in the TWE program, and 39 to the control group, which could not participate in TWE but could receive JP services. JP services included a review of job and employment goals with program staff and referrals to available job-placement services and resources, including Internet job listings, computer access, and support in creating a resume. The authors collected data on study participants’ demographics, work histories, and clinical histories through baseline questionnaires; they also collected outcome data on employment and income through follow-up questionnaires at 6 and 12 months following program enrollment. The authors also accessed VHA medical records to confirm use of substance abuse services and psychiatric hospitalizations. To assess potential impacts of the TWE program, the authors compared the unadjusted differences between the treatment and control groups on various employment outcomes.
Findings
- The study found that the TWE group worked significantly more total hours than the control group over the one-year follow-up period for all paid employment, which included positions from both competitive and TWE-subsidized employment. The study also found that the treatment group worked significantly more total weeks in any paid employment over the one-year follow-up period than the control group (28 versus 18 weeks). Finally, in the one year follow-up period, the treatment group earned significantly more total income from any paid employment than the control group ($10,353 versus $6,581).
Considerations for Interpreting the Findings
Ten members of the control group received TWE services before final data collection. The authors stated that data for these people were censored at the point that they entered treatment services. However, it is not clear how their follow-up data were used in the impact analysis, meaning attrition from the randomized sample cannot be calculated given the information in the study.
The significant, positive findings for earnings and employment in any paid employment positions could largely have been driven by program participation, as income resulting directly from involvement in TWE was counted in the any paid employment outcomes.
Finally, the authors estimated multiple related impacts on outcomes related to employment. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in this domain is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not provide enough information to calculate attrition and did not include sufficient controls in their analysis to establish that treatment and control groups were similar on observable characteristics at baseline for the analytic sample. This means we are not confident that the estimated effects are attributable to TWE. Other factors are likely to have contributed.