Skip to main content
UNITED STATES DEPARTMENT OF LABOR

BOND implementation and evaluation: 2017 stage 1 interim process, participation, and impact report (Croake et. al, 2017)

Review Guidelines

Causal Evidence Guidelines

Absence of conflict of interest.

Citation

Croake, S., Mann, D., Anad, P., Geyer, J., Jones, C., Gubits, D., Morrison, K., Bell, S., Orfield, C., McGuirk, A., Stapleton, D., Rose, T., Hoffman, D., & Wittenburg, D. (2017). BOND implementation and evaluation: 2017 stage 1 interim process, participation, and impact report. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research

Highlights

  • The study's objective was to examine the impact of the Benefit Offset National Demonstration (BOND) on earnings and public benefit receipt. 
  • The study was a randomized controlled trial. Using administrative data, the authors conducted statistical models to compare the outcomes of the treatment and control group members. 
  • The study found that BOND intervention participation was significantly related to higher total amounts of Social Security Disability Insurance (SSDI) benefits.  
  • This study receives a low causal evidence rating. This means we are not confident that the estimated effects are attributable to BOND; other factors are likely to have contributed. 

Intervention Examined

The Benefit Offset National Demonstration

Features of the Intervention

Following the 1999 Work Improvement Act and Ticket to Work (TTW), the Social Security Administration (SSA) began testing alternatives to Social Security Disability Insurance (SSDI) work rules. These changes were intended to increase the gainful employment for SSDI beneficiaries and reduce their need for benefits by incentivizing working. The Benefit Offset National Demonstration (or BOND) is one such initiative which has been testing incentives tied to annual earnings through randomized controlled trials. The intervention was delivered by a variety of public benefits staff across the 10 participating study sites including program administrators, supervisors, benefits counselors, and field staff. The BOND intervention is designed to serve the entire population receiving SSDI benefits.

Features of the Study

This study was a randomized controlled trial that examined how national benefit offset programs and changes to related supports affected earnings and SSDI benefit outcomes for the entire SSDI service population. Ten sites were randomly selected across the nation to provide a representative sample of SSDI beneficiaries. Study sites were randomized by computer, enrolling a total of 79,436 beneficiaries into the treatment group. The treatment group received Work incentives Counseling (WIC) customized to their benefits and the $1 and $2 benefit offset as designed. The control group continued services as usual, which consisted of no cash incentive and Work Incentives Planning Assistance Counseling (WIPA). While the counseling programs differed, they were delivered at similar levels of intensity. The study used a statistical model to compare the outcomes of treatment and control group members across the 10 sites as reported in administrative data. Data were sourced from the Social Security Administration's Payment History Update System (PHUS) for SSDI, the Supplemental Security Record for SSI, and the SSA Master Earnings File and included earnings and benefits paid during the 2015 calendar year.  

Study Sites

  • Alabama 
  • Arizona/Southeastern California 
  • Colorado/Wyoming 
  • DC Metro 
  • Greater Detroit 
  • Greater Houston  
  • Northern New England 
  • South Florida 
  • Western New York 
  • Wisconsin 

Findings

Earnings and wages

  • The study found no statistically significant relationships between participation in the BOND program and earnings.  

Public benefits receipt

  • The study found a statistically significant relationship between participation in the BOND program and higher total SSDI benefits.  

Considerations for Interpreting the Findings

Although the study was a randomized controlled trial, the authors did not provide adequate information to calculate attrition. Due to the unknown attrition rate, the study was reviewed under the regression guidelines. The authors did not account for preexisting differences between the groups before intervention participation. These preexisting differences between the groups—and not the intervention— could explain the observed differences in outcomes. Therefore, the study is not eligible for a high or moderate rating.  

Causal Evidence Rating

The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to BOND; other factors are likely to have contributed.  

Reviewed by CLEAR

November 2023

Topic Area