Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Fraker, T., Mamun, A., Manno, M., Martinez, J., Reed, D., Thompkins, A., & Wittenburg, D. (2012). The Social Security Administration’s Youth Transition Demonstration Projects: Interim report on West Virginia Youth Works. Princeton, NJ: Mathematica Policy Research.
Highlights
- The study’s objective was to measure 12-month impacts of the West Virginia Youth Works project, one of six project sites of the larger Social Security Administration (SSA)-sponsored Youth Transition Demonstration (YTD) that used a randomized evaluation. Youth Works provided services to help youth with disabilities improve their education and employment outcomes and become more economically self-sufficient.
- Data were collected from a baseline survey and 12-month follow-up survey, the Ticket Research File (TRF), and the Master Earnings File (MEF). The authors estimated program impacts on measures of paid employment, educational attainment, and Social Security disability benefit receipt.
- The study found that Youth Works had positive impacts on employment, earnings, and total income after one year of implementation. There were no impacts on educational attainment and Social Security disability benefit receipt.
- The quality of causal evidence presented in the study is high for the education, employment, and SSA benefit receipt outcomes because they are based on a well-implemented randomized controlled trial with low attrition. However, the quality of causal evidence is moderate for earnings and total income because sample attrition for these outcomes was high.
Intervention Examined
The Youth Transition Demonstration Projects, West Virginia Youth Works Program
Features of the Intervention
The West Virginia Youth Works program was one of six project sites evaluated using a randomized design as part of the larger SSA-sponsored YTD, which intended to help youth with disabilities become more self-sufficient and improve their employment outcomes. From April 2008 to March 2012, the Human Resource Development Foundation—a nonprofit employment services provider—collaborated with the Center for Excellence in Disabilities to offer work-readiness assessments and services, job placement, and case management services to young SSA recipients (ages 14 to 25).
All YTD participants were also eligible for waivers that (1) extended the student earned income exclusion to all YTD participants regardless of age, (2) increased the earned income exclusion to a $1 reduction in Supplemental Security Income benefit for every $4 earned above a base amount, and (3) delayed benefit cessation for YTD participants who were determined ineligible for benefits after a benefit review or age-18 SSI medical redetermination.
Features of the Study
The study was a randomized controlled trial in which 455 youth were randomly assigned to the treatment group and 397 to the control group. Data were collected from baseline and 12-month follow-up surveys and SSA’s TRF and MEF. The TRF supplied data on benefit receipt status, benefit amounts paid, and disabling condition, whereas the MEF provided baseline earnings information. Regression-adjusted impacts were estimated for several outcomes, such as paid employment, earnings, educational attainment, monthly Social Security disability benefit receipt, and total income from earnings and benefits.
Findings
- The study found that Youth Works had positive impacts on employment and earnings in the first 12 months of follow-up. Participants were 19.1 percentage points more likely to have been employed in the past year and earned $524 more, on average, than those in the control group. Average annual income, measured as SSA benefits and earnings combined, for members of the Youth Works group was $717 greater for treatment participants than for the control group.
- The study found no significant impacts on education or Social Security disability benefit receipt.
Considerations for Interpreting the Findings
About one-third of study participants were still in school during the study period and, therefore, had limited employment opportunities. In addition, only 85 percent of youth assigned to the treatment group actually enrolled in Youth Works. The authors imputed earnings-related outcome variables for some of those included in the analysis for whom those outcomes were missing. Because the study had high attrition for these outcomes, imputation could influence the results in unknown ways.
Causal Evidence Rating
The quality of causal evidence presented in the study is high for the education, employment, and SSA benefit receipt outcomes because they are based on a well-implemented randomized controlled trial with low attrition. This means we are confident that estimated effects on these outcomes are attributable to Youth Works and not to other factors. The quality of causal evidence is moderate for earnings and total income because sample attrition for these outcomes was high, but the authors controlled for key differences between the treatment and control groups at baseline. This means we are somewhat confident that estimated effects on these outcomes are attributable to Youth Works, although other factors might also have contributed.