Frey, W., Drake, R., Bond, G., Miller, A., Goldman, H., Salkever, D., & Holsenbeck, S. (2011) Mental Health Treatment Study: Final report. Report submitted to the Social Security Administration. Rockville, MD: Westat.
- The study’s objective was to examine the implementation and impact of the Mental Health Treatment Study, which provided individual placement and support (IPS, an evidence-based model of supported employment services) and systematic medication management to Social Security Disability Insurance (SSDI) recipients with psychiatric disabilities. This profile focuses on the implementation study; see CLEAR’s profile of the impact study here.
- The implementation study focused on fidelity to the program model, consistency in service delivery, where services occurred, and the relationship between fidelity and employment rates. The authors collected data from regular telephone calls with program staff, site visits, participants’ medical records, follow-up interviews, and service use data. To measure fidelity, the study used a 15-point IPS fidelity scale; each scale item reflected a specific element in the IPS practice.
- Overall, the study found that 77 percent of sites achieved high fidelity in the first year of implementation and 86 percent of sites achieved high fidelity in the second and third years. More than 90 percent of participants engaged in some type of supported employment service in the first reporting period and almost 90 percent did so in the second reporting period.
- Study findings related to implementation challenges and solutions are potentially applicable to other current or future projects that provide employment-related services to people with psychiatric disabilities.
The Mental Health Treatment Study (MHTS)
Features of the Intervention
The Mental Health Treatment Study tested an IPS supported employment intervention coupled with systematic medication management. Supported employment models provide a combination of employment and health services and supports to improve employment and other outcomes for people with psychiatric conditions. Programs in the study provided a bundled, comprehensive intervention that combined evidence-based mental health treatments and services with integrated employment services. The program lasted 24 months and included supportive employment services, systematic medication management, behavioral health services, reimbursement for out-of-pocket behavioral health expenses, and coordinated mental health services. In addition, SSDI medical continuing disability reviews were suspended for three years from the date of program enrollment for those receiving the intervention. After 20 months in the program, participants began a four-month transition process to help return to normal services. Program implementation occurred from 2006 to 2010.
Features of the Study
The implementation portion of the study sought to describe the extent to which the 23 participating sites implemented the service intervention as intended. In doing so, the study described the major aspects of implementation, including contextual factors, planning, and design; inputs and resources, including staffing; and service delivery, including engagement with services and dosage. The study drew on data from weekly telephone calls (intended to provide ongoing feedback and technical assistance) with program staff, two site visits, participant medical records, follow-up interviews with participants, and service use data from a quality management template completed by program staff. The study was conducted from November 2006 through July 2010, with site visits occurring in January 2008 and January 2009.
- Context, planning, and design. The study operated in a challenging economy. Most sites faced severe financial stress due to the economic recession and state funding cutbacks, and two sites ceased recruitment and enrollment activities within the first year. Despite these challenges, the community mental health centers recruited for the study were already familiar with the supported employment model, able to start operations quickly, and negotiated with other agencies for the provision of services the centers could not provide directly. The characteristics of some participants also presented challenges to the program design. For instance, program staff noted a considerable lack of engagement on the part of unemployed participants and found that some participants were too disabled to wait for services at the centers. Physical health impairments and substance abuse problems among participants also posed great difficulties to efforts to increase employability of participants.
- Inputs and resources. Each study site employed a nurse care coordinator who supported systematic medication management and medical and psychiatric care; a supported employment specialist, who focused on providing employment supports; and a research assistant, who recruited and enrolled participants and administered post-baseline participant interviews. The study also employed quality management program directors to provide ongoing technical assistance and feedback to sites to promote adherence to the IPS model.
- Service delivery and engagement. Engagement with different types of services varied widely across sites. On average, more than 90 percent of participants completed vocational plans and assessments; 69 percent received some form of benefits counseling; 54 percent received case management; 53 percent received general medical care; and fewer than a quarter received social skills training, financial assistance, housing assistance, substance abuse treatment, family counseling, or legal assistance. The study found the lack of some of these behavioral support services (particularly case management) and the inability to offer services in-house to be an implementation challenge. However, program staff sometimes took on the case management role if it was not otherwise available. Participants’ contact with their supported employment specialists and employers declined over time.
More than 90 percent of participants engaged with the systematic medication management component of the intervention and 57 percent were always engaged. More than 93 percent of participants reported that they took their medicine as prescribed most of the time and during the final follow-up interview; more than two-thirds expressed a positive attitude toward the use of psychotropic medications to control their symptoms. The off-site location of many medication prescribers presented difficulties in integrating this component with the rest of the intervention, but implementing staff filled in gaps on prescribers’ reports or aggressively pursued prescribers to supply these data.
- Fidelity to the program model. The study found that the sites generally implemented the supported employment model with fidelity, with 77 percent of them achieving high fidelity in the first year and 86 percent of them achieving high fidelity in the second and third years. Overall, fidelity ratings averaged 67.6 in Year 1, 69.3 in Year 2, and 67.2 in Year 3, out of a total possible score of 75, based on a 15-point IPS fidelity scale in which each scale item reflected a specific element in the supported employment intervention. However, among the program participants, fidelity ratings did not seem to be associated with higher employment rates (as reported by the participants themselves).
Considerations for Interpreting the Findings
The study clearly described how it derived fidelity and implementation measures, and it illustrated findings with descriptive quantitative service receipt data. The study would have benefited from including a more thorough description of on-site data collection. In addition, for the study the authors selected from among 50 sites that were already using an IPS model; they chose not to do random selection through a nationally representative probability sample because of the need for quick start-up with all program components in place. Although the authors suggested that the sites were selected to support generalizability of the findings, it is unclear to what extent the study findings would apply to future projects that provide employment-related services to people with psychiatric disabilities.