Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Fraker, T., Baird, P., Mamun, A., Manno, M., Martinez, J., Reed, D., & Thompkins, A. (2012). The Social Security Administration’s Youth Transition Demonstration Projects: Interim report on the Career Transitions Program. Princeton, NJ: Mathematica Policy Research.
Highlights
- This study’s objective was to measure 12-month impacts of the Maryland Career Transitions Program (CTP), one of six project sites of the larger Social Security Administration (SSA)-sponsored Youth Transition Demonstration (YTD) projects that used a randomized evaluation. CTP provided services to help youth with severe emotional disturbances or significant mental illness improve their education and employment outcomes and become more self-sufficient.
- Data were collected from a baseline survey and 12-month follow-up survey, the Ticket Research File (TRF), and the Master Earnings File (MEF). The authors estimated program impacts on measures of paid employment, educational attainment, and Social Security disability benefit receipt.
- The study found that CTP did not achieve any statistically significant impacts on employment, earnings, total income, or Social Security disability benefit receipt during the 12-month follow-up period. However, CTP participants were more likely to enroll in postsecondary education than members of the control group.
- The quality of causal evidence presented in the study is high for the education and SSA benefit receipt outcomes because they are based on a well-implemented randomized controlled trial with low attrition. However, the quality of causal evidence is moderate for employment, earnings, and total income because sample attrition for these outcomes was high.
Intervention Examined
The Youth Transition Demonstration Projects, Career Transitions Program
Features of the Intervention
Implemented from April 2008 to January 2011, CTP provided employment- and education-related, benefits planning, and case management services to youth and young adults with severe emotional disturbances or significant mental illness. It was one of six project sites evaluated using a randomized design as part of the larger SSA-sponsored YTD, which intended to help youth with disabilities become more self-sufficient and improve their employment outcomes. CTP was delivered by St. Luke’s House, a community mental health services provider in Montgomery County, Maryland.
The program targeted high school juniors, seniors, and recent exiters with severe emotional disturbances (schizophrenia, personality, mood, conduct, anxiety, attention deficit, attention deficit hyperactivity, and depression disorders) or other significant mental illness. All youth were currently receiving or considered at high risk of eventually receiving disability benefits. All YTD participants were also eligible for waivers that (1) extended the student earned income exclusion to all YTD participants regardless of age, (2) increased the earned income exclusion to a $1 reduction in Supplemental Security Income (SSI) benefit for every $4 earned above a base amount, and (3) delayed benefit cessation for YTD participants who were determined ineligible for benefits after a benefit review or age-18 SSI medical redetermination.
Features of the Study
The study was a randomized controlled trial that assigned 422 youth to the treatment group and 383 to the control group. Data were collected from a baseline survey, a 12-month follow-up survey, and the SSA’s TRF and MEF. The TRF supplied data on benefit receipt status, benefit amounts paid, and disabling condition, whereas the MEF provided baseline earnings information. Regression-adjusted impacts were estimated for several outcomes, such as paid employment, earnings, educational attainment, monthly Social Security disability benefit receipt, and total income from earnings and benefits.
Findings
- CTP achieved no statistically significant impacts on paid employment, earnings, total income, or Social Security disability benefit receipt.
- Members of the CTP group were 8 percentage points more likely to enroll in postsecondary education than members of the control group.
Considerations for Interpreting the Findings
The authors attributed the lack of impacts to the relatively low employment barriers faced by participants and the relatively rich services available to the control group. The authors imputed earnings-related outcome variables for some of those included in the analysis for whom those outcomes were missing. Because the study had high attrition for these outcomes, imputation could influence the results in unknown ways.
Causal Evidence Rating
The quality of causal evidence presented in the study is high for the education and SSA benefit receipt outcomes because they are based on a well-implemented randomized controlled trial with low attrition. This means we are confident that the estimated effects on these outcomes are attributable to the CTP and not to other factors. The quality of causal evidence is moderate for employment, earnings, and total income because sample attrition for these outcomes was high, but the authors controlled for key differences between the treatment and control groups at baseline. This means we are somewhat confident that estimated effects on these outcomes would be attributable to the CTP; however, the study found no impacts on these outcomes.