In recent decades, government expenditures for safety nets for people with disabilities—such as income support from the Supplemental Security Income (SSI) or Social Security Disability Insurance (SSDI) programs—have increased (Houtenville & Ruiz 2012; Livermore et al. 2011). For example, SSDI, which provides income support to workers with disabilities and their families, has grown from 2.8 million beneficiaries in 1980 to 10.9 million beneficiaries in May 2015 and now pays a total monthly benefit of $11.1 billion (Social Security Administration [SSA] 2009; 2015). The Disability Insurance Trust Fund, from which all SSDI benefits are paid, is projected to be depleted by late 2016 (SSA, Office of the Chief Actuary 2014). In response to these trends, several initiatives have sought to improve the employment and earnings outcomes of these populations and thereby decrease their need for SSI or SSDI benefits or other government assistance.
CLEAR conducted a systematic literature search to identify all the studies of employment initiatives from 1985 to May 2014 that targeted SSI or SSDI beneficiaries or people identified as being at risk of entering those programs and then determined the strength of causal evidence presented in each (see the CLEAR’s Process box for details). We present a synthesis of findings from the studies that provide high or moderate causal evidence based on this review in the first section of this brief. We also reviewed implementation reports of the demonstrations with high or moderate causal evidence ratings to provide lessons for developing future interventions to serve people with disabilities, which we discuss in the second section of this brief.
Evidence echoes previous literature reviews on the challenges of generating substantive impacts, though customized supports to well-targeted populations show some potential.
The conclusions from CLEAR’s systematic literature search and review process largely echo the key findings from Wittenburg et al. (2013), which summarized the existing literature on employment-focused interventions for people with disabilities. Overall, interventions that provided intensive employment support services and/or employment incentives had moderate success improving employment and earnings outcomes but did not decrease disability income support payments.
The most effective interventions provided intensive, customized supports and services focused on job training, placement, and retention to narrowly defined target populations.
In the Youth Transition Demonstration (YTD), which targeted transition-age youth, the projects that focused their efforts on direct employment services (including outreach to employers, job shadowing, and direct placement) had positive effects on youths’ earnings and employment outcomes, whereas those that focused on case management (including identifying goals, managing time, and connecting to social and health services) had none (Fraker et al. 2014). Similarly, when Kornfeld and Rupp (2000) examined impacts for Project NetWork participants who received employment-focused case-management services, they found that the impacts were smallest for the least service-intensive model. Frey et al. (2011) found that the Mental Health Treatment Study, which targeted SSI and SSDI beneficiaries with psychiatric impairments, improved several employment, earnings, and health outcomes for treatment group members.
More generally, target populations experiencing the largest effects included people with psychiatric disabilities (Cook et al. 2008; Frey et al. 2011), people with developmental disabilities (Kerachsky & Thornton 1987; Decker & Thornton 1995), and youth (Fraker et al. 2014). In contrast, interventions that did not target people with specific impairments, such as the Ticket to Work (TTW) program, which mails a voucher for employment services to all SSI and SSDI beneficiaries that they can use voluntarily, had relatively smaller impacts (Stapleton et al. 2013a).
Interventions that provided support services or incentives to help beneficiaries keep more of their benefits when working had small or no impacts on employment, even if spending on services was high.
Examples of projects with limited impacts on employment and earnings include the Accelerated Benefits Demonstration (ABD), Benefit Offset Pilot Demonstration (BOPD), and TTW program (Weathers & Bailey 2014; Weathers & Hemmeter 2011; Stapleton et al. 2013a). In each case, the interventions had limited success improving employment and earnings outcomes despite substantial costs associated with them.
There is no evidence of SSI or SSDI caseload reductions, even among interventions that improved employment and/or earnings.
The programs and demonstrations reviewed did not achieve a key objective—increasing the participants’ earnings enough to decrease their benefit receipt. For example, in four YTD projects, treatment group members’ SSI receipt increased two years after random assignment (Hemmeter 2014). The increases were due to SSI program waivers at those four projects that protected the participants’ benefit receipt status and benefit amounts. However, if the YTD is to ever achieve SSI program savings, then YTD participants’ receipt of SSI benefits will eventually have to decrease.
The Benefit Offset National Demonstration (BOND), which is testing the provision of work incentives and other supports, provides another example. BOND’s benefit offset replaces the complete loss of all benefits for working SSDI beneficiaries, instead gradually decreasing the SSDI benefit by $1 for every $2 earned above the substantial gainful activity amount. If the benefit offset is to decrease total SSDI benefits paid to BOND participants, then enough BOND participants must respond to the benefit offset by increasing their earnings enough to partially decrease their SSDI benefit. However, BOND did not generate impacts on employment or earnings in its first year of operations (Stapleton et al. 2013b), though several factors suggest that positive impacts on earnings might yet emerge.
Little is known about interventions for improving earnings of people with TBI and PTSD.
Our review found only four studies examining the effectiveness of interventions for people with TBI or PTSD on their return to work, and only one of these examined both employment and earnings outcomes. All four of these studies focused on military veterans.
Davis et al. (2012) randomly assigned 85 volunteer veterans with PTSD to receive services from either Individual Placement and Support (IPS)—a supported employment model—or the Department of Veterans Affairs’ standard Vocational Rehabilitation Program, which provided work therapy through setaside temporary jobs. Veterans who received IPS were significantly more likely to gain competitive employment, worked in a competitive job more weeks, and earned more during the 12-month follow-up period. However, the IPS recipients’ total income, on average, was still below self-sufficiency levels.
Twamley et al. (2014) examined the impact of supplementing supported employment services with cognitive training, finding a doubling in employment rates for those who received such training, but earnings impacts were not examined. Salazar et al. (2000) and Vanderploeg et al. (2008) compared different types of rehabilitation programs for veterans with TBI and found no differences in their return to work or military duty. These two studies did not report earnings impacts.
Recruiting beneficiaries to participate in demonstrations was difficult, which limited the generalizability of study findings.
Most SSA employment demonstrations have struggled to recruit volunteer participants. With a few exceptions, the interventions tested targeted people for services after they had met SSA’s disability criteria and started receiving benefits. To become eligible for SSA disability benefits, applicants must prove that their impairments make it impossible to work at substantive levels. It is therefore unlikely that beneficiaries who have gone through the application process will volunteer for programs that promote work, for fear of losing benefits. Although some interventions used program waivers, such as allowing beneficiaries to keep more of their benefits while working, participants still could lose some benefits by increasing their earnings.
For studies that rely on volunteers, the generalizability of study findings to the entire study recruitment pool depends in part on what percentage of the recruitment pool volunteered for the study. The smaller the volunteer group, the greater the concern that the volunteers were not representative of the larger group. Most of the demonstration projects SSA funded enrolled about 5 percent of the population targeted for recruitment (Rangarajan et al. 2008).
More recent demonstrations with narrow target populations of youth and those with psychiatric impairment had higher participation rates. For example, the YTD projects used all available tools and resources and worked very hard to achieve evaluation enrollment rates ranging from 16 to 30 percent of eligible youth (Fraker et al. 2014). The ABD, which had a participation rate of 99 percent (Michalopoulos et al. 2011).
The ABD provided health insurance coverage as its primary intervention—only a subset of ABD treatment group members received employment supports. ABD’s high participation rate was due to the strong demand for free health insurance coverage among the target population—SSDI beneficiaries without health insurance who were in the 24-month Medicare waiting period. The ABD evaluation revealed that those who volunteered for the demonstration often had unmet medical needs and that the intervention helped address those needs.
Fidelity to the demonstration model is important.
Several different studies provide evidence that favorable impacts are more likely to emerge when the demonstration model is closely followed. Programs that strictly adhere to the IPS model have shown significant impacts on employment and earnings of people with psychiatric impairments (Cook et al. 2005). Specifically, models that integrated vocational services and clinical mental health services, such as medication management and individual therapy, were more effective than models with low levels of service integration, such as those in which vocational rehabilitation and clinical counseling were provided by separate agencies or in separate locations.
Additionally, Fraker et al. (2014) found that YTD projects that were implemented with fidelity to the YTD program model were more effective than programs that were not. These evaluations included detailed documentation of the services delivered to ensure the findings could be replicated in other settings.
Work incentives and supports can be difficult to implement in the context of SSA’s existing work incentives, creating potential confusion for beneficiaries and program staff.
SSA’s complex eligibility determination processes can make it challenging to implement new interventions or approaches that administrators and staff can readily understand. For example, the TTW program has a complex system of incentives that has failed to produce positive outcomes. The TTW program provides SSI and SSDI beneficiaries with more choices of employment services vendors and offers employment-support service providers financial incentives to serve beneficiaries who reach earnings milestones. However, many consider the payment system complex and cumbersome and find it difficult to determine when beneficiaries reach the milestones that generate provider payments; as a result, it has been difficult to recruit providers (Stapleton et al. 2013a).
Similarly, BOND, which had very few participants during its first year, was implemented with other complex, existing work incentives (Stapleton et al. 2013b). For example, the benefit offset is provided only after the Trial Work Period ends and SSA staff have completed a Work Continuing Disability Review to evaluate the beneficiary’s work effort and continued eligibility for benefits.
A strong technical assistance component, with incentives for service providers to accept the assistance, is important to successful implementation.
From the outset of the YTD, the technical assistance that YTD projects received was geared toward achieving desirable employment outcomes for project participants. However, the process analysis of the three projects implemented early in YTD (Phase 1) revealed a need to focus the technical assistance on services directly linked to paid employment and to closely monitor both the delivery of those services and participants’ outcomes. Technical assistance for the three projects implemented later (Phase 2) was adjusted accordingly and helped the Phase 2 projects focus more closely on connecting youth with competitive paid jobs.
For several projects, technical assistance provided under the evaluation contract greatly facilitated the delivery of employment services. For example, at one project site, quantitative data were used during the intervention period to identify program staff whose caseloads were not meeting program targets, and those staff then received opportunities for professional development.
Funders and operators of future interventions with objectives and target populations similar to those of YTD should consider offering service providers high quality technical assistance on the design and delivery of employment services (Fraker et al. 2014).
Demonstrations should be pilot tested before being implemented on a national scale.
In reviewing the implementation of the TTW program, the U.S. Government Accountability Office (GAO 2004) argued that the rush to implement the program created inefficiencies that could have been addressed in a smaller pilot. GAO claimed that if SSA had tested various components of the TTW program before launching it nationwide, it might have identified problems and developed solutions before implementation. In 2008, SSA revised the TTW program regulations to address some of these initial shortcomings.
The benefits of developing a pilot program before launching a major demonstration were illustrated by SSA’s BOPD, which was the precursor to the larger, ongoing BOND. The pilot demonstration was implemented in four states to test the administrative processes needed for BOND. The original plans for implementing BOND were modified based on experiences gleaned from the pilot demonstrations (Bell et al. 2011).