There is no conflict of interest.
Citation
Mamun, A., Wittenburg, D., Denny-Brown, N., Levere, M., Mann, D., Coughlin, R., Croake, S., Gordon, H., Hoffman, D., Holzwart, R. & Keith, R. (2021). Promoting Opportunity Demonstration: Interim Evaluation Report. Washington, DC: Mathematica.
Highlights
- The study's objective was to examine the impact of the Promoting Opportunity Demonstration (POD) on earnings, employment, and public benefits receipt.
- This study used a randomized controlled trial design that assigned eligible Social Security Disability Insurance (SSDI) beneficiaries to the treatment or control groups. Using administrative and survey data, the authors conducted statistical models to compare the differences in outcomes between the groups.
- The study found no significant differences in earnings, substantive employment, annual SSDI benefit amounts, and total annual income between the treatment and control groups.
- This study receives a high evidence rating. This means we are confident that any estimated effects are attributable to the Promoting Opportunity Demonstration (POD), and not to other factors. However, the study did not find statistically significant effects.
Intervention Examined
The Promoting Opportunity Demonstration (POD)
Features of the Intervention
The Promoting Opportunity Demonstration (POD) tested new benefit offset rules for Social Security Disability Insurance (SSDI) beneficiaries. The POD was implemented to address complexities associated with the SSDI by using a benefit offset ramp to replace SSDI's cash cliff: a situation where SSDI beneficiaries who work and earn a single dollar over the substantial gainful activity (SGA) threshold after a three-month grace period could lose all cash benefits through suspension and termination. The POD replaced the cash cliff with a benefit offset which reduced benefits by $1 for every $2 earned above the higher of the threshold. In addition, the POD offered direct support services to beneficiaries, including a call center and counseling.
There were two versions of the POD rules. In the first version, beneficiaries could earn above the threshold without losing their SSDI entitlement, even if their benefits were reduced to zero. If their earnings later dropped, their benefits could resume. The second version allowed for the loss of SSDI entitlement if beneficiaries remained in full offset for 12 consecutive months. In the current study, the authors combined recipients of both versions to assess the overall impact of the new rules.
Features of the Study
The study was a randomized controlled trial that assigned working-age SSDI beneficiaries to the treatment or control groups. Eligible participants included SSDI beneficiaries who lived in a POD implementation area, were eligible for POD, and received a primary mailing as part of direct outreach for POD. Of the 10,070 eligible participants, 6,700 were randomly assigned to the treatment group and 3,370 were randomly assigned to the control group. The treatment group included POD enrollees whose benefits were subject to POD rules, and they either did or did not face termination if they had 12 consecutive months of earnings above the full offset amount. The control group included POD enrollees whose benefits were adjusted under the current SSDI rules, not POD rules. The average age of the study sample was 47.4 years, 55% of the sample were women, 40% were non-Hispanic White, and 47% had a high school diploma or GED.
The study used a combination of program records data from the Social Security Administration (SSA) as well as the POD baseline and one-year follow-up surveys. The authors used statistical models to compare the differences in earnings, employment, and public benefits receipt between the treatment and control group members.
Study Sites
- Alabama
- California
- Connecticut
- Maryland
- Michigan
- Nebraska
- Texas
- Vermont
Findings
Earnings and wages
- The study found no significant differences in earnings between the treatment and control groups. The study also found no significant differences between the groups in total annual income including earnings, SSDI benefits, and Supplemental Security Income.
Employment
- The study found no significant differences in substantive employment between the treatment and control groups.
Public benefits receipt
- The study found no significant differences in annual SSDI benefit amounts between the treatment and control groups.
Considerations for Interpreting the Findings
The authors conducted intent-to-treat analyses which estimates the effects of POD rules on treatment group members regardless of whether they used the benefit offset or not. As a result, the program’s impact on individuals who actually participated might be higher than the study’s estimates. Also, the authors consider p-values of less than 0.10 to be significant, though it is standard practice to consider results as statistically significant if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well implemented randomized controlled trial. This means we are confident that any estimated effects would be attributable to the Promoting Opportunity Demonstration (POD) and not to other factors. However, the study did not find statistically significant effects.