Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Hemmeter, J. (2014). Earnings and disability program participation of Youth Transition Demonstration Participants after 24 months. Social Security Bulletin, 74(1), 1–26.
Highlights
- This study’s objective was to measure 24-month impacts for the six Youth Transition Demonstration (YTD) sites that used a randomized evaluation. YTD projects provided services to help youth with disabilities improve their education and employment outcomes and become more economically self-sufficient.
- Data were collected from the Master Earnings File (MEF), Supplemental Security Record (SSR), Master Beneficiary Record (MBR), and Numerical Identification System (Numident) file. The authors estimated program impacts on measures of paid employment and Social Security disability benefit receipt.
- The study found that, in three of the sites examined, YTD participants were more likely to have at least some earnings in the first calendar years after random assignment; this was also the case for three sites in the second calendar year after random assignment. However, YTD had no statistically significant impacts on average earnings during either calendar year. In four of the six sites, YTD participants were more likely to receive Supplemental Security Income (SSI) 24 months after random assignment.
- The quality of the causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we can be confident that the estimated impacts are attributable to YTD, and not other factors.
Intervention Examined
The Youth Transition Demonstration
Features of the Intervention
Sponsored by the Social Security Administration (SSA), the YTD projects were intended to help youth ages 14 to 25 with disabilities who received or were at risk of receiving Social Security disability benefits become more economically self-sufficient and improve their employment outcomes. Interventions and target populations varied across sites, but those assigned to the YTD treatment groups were offered some combination of benefits counseling, career counseling, and coordination of services, as well as internships, job shadowing, job coaching, competitive paid employment, and empowerment training.
All YTD participants were also eligible for waivers that (1) extended the Student Earned Income Exclusion to all YTD participants, regardless of age, who attended school; (2) increased the Earned Income Exclusion to a $1 reduction in SSI benefit for every $4 earned above a base amount; and (3) delayed benefit cessation for YTD participants who were determined ineligible for benefits after a continuing disability review or age-18 SSI medical redetermination.
Features of the Study
Six YTD sites were evaluated using a randomized controlled trial at each site. The analysis reported in this study relied exclusively on SSA administrative data, although interim reports also relied on survey data. Evaluation data sources included the MEF for annual earnings data; the SSR and MBR for SSI and Social Security Disability Insurance (SSDI) program participation information, respectively; and the Numident file for dates of death. The author estimated the impacts on paid employment, earnings, and Social Security disability benefit receipt.
Study Sites
- Colorado Youth WINS (CO)
- City University of New York YTD Project (Bronx, NY)
- Erie County Transition WORKS (Erie County, NY)
- Montgomery County Career Transition Program (Montgomery County, MD)
- Florida Miami-Dade County Broadened Horizons, Brighter Futures (Miami-Dade County, FL)
- West Virginia Youth Works (WV)
Findings
- In three of the six sites (Bronx, NY; FL; WV) YTD participants were 6 to 25 percentage points more likely to have any earnings in the first full calendar year post random assignment.
- In three of the six sites (Bronx, NY; CO; WV) YTD participants were 6 to 10 percentage points more likely to have any earnings in the second full calendar year post random assignment.
- In four of the six sites (Bronx, NY; Erie, NY; FL; WV) YTD participants were 6 to 12 percentage points more likely to receive SSI and 6 to 11 percentage points more likely to receive either SSI or SSDI two years after random assignment.
- None of the sites had statistically significant impacts on paid employment or average earnings during either the first or second full calendar year after random assignment.
Considerations for Interpreting the Findings
Because the MEF reports earnings by calendar year, employment and earnings impacts could not be estimated for the months in the calendar year of random assignment.
The study authors estimated multiple related impacts on outcomes of interest separately for each of six sites. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings from the study is likely to be overstated.
Causal Evidence Rating
The quality of the causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we can be confident that the estimated impacts are attributable to YTD, and not other factors.