Citation
Tremblay, T., Smith, J., Xie, H., & Drake, R. (2006). Effect of benefits counseling services on employment outcomes for people with psychiatric disabilities. Psychiatric Services, 57(6), 816-821.
Highlights
- The studies’ objective was to measure the earnings impacts of the Vermont Work Incentive Initiative (VWII), which provided customized benefits counseling to Social Security disability benefit recipients who were receiving vocational rehabilitation (VR) services. Services offered by the VWII participants varied, but could consist of benefits screening, explanations of work incentives, and case management.
- The studies used earnings data collected from the Vermont Department of Employment and Training. The treatment groups received VWII services and possibly other VR services, whereas the comparison groups received VR services excluding benefits counseling services. The quasi-experimental evaluation employed matched comparison groups and measured impacts using a mixed-effects linear model.
- VWII was correlated with significant increases in mean earnings for the treatment group—averaging about $1,250 per year among participants with psychiatric disabilities and about $760 to $900 per year among all participants.
- These studies receive a low causal evidence rating because they did not adequately demonstrate baseline equivalence between the treatment and comparison groups or control in the analysis for key factors. A low causal evidence rating means that we are not confident that the estimated effects are attributable to the VWII and other factors likely played a role.
Intervention Examined
The Vermont Work Incentive Initiative
Features of the Intervention
VWII began in 1999 and provides specialized benefits counseling to Social Security Disability Insurance (SSDI) or Supplemental Security Income (SSI) benefit recipients who are also receiving VR services. The intensity and duration of the counseling services provided varies, but the services generally consist of benefits screening (researching the recipient’s current benefits status and potential employment issues); providing information about how employment would affect the recipient’s benefits and available work incentives; and assistance in developing a benefits management plan.
The studies evaluated the VWII’s first three years—from 1999 to 2002. The 2004 study examined outcomes among all VWII participants, whereas the 2006 study focused exclusively on outcomes for VWII participants with psychiatric disabilities. For each study, the treatment group consisted of SSDI or SSI benefit recipients who received VWII services and potentially other VR services, whereas the comparison group consisted of SSDI or SSI benefit recipients who received VR services excluding benefits counseling services—that is, they received no VWII services. All study participants were between the ages of 18 and 60 and actively looking for work. According to VR, approximately 48 percent of VWII study participants had a psychiatric condition as their primary disability.
Features of the Study
To implement their quasi-experimental evaluations, each study’s authors constructed two matched comparison groups. One comparison group—the “contemporaneous” comparison group—consisted of the Division of Vocational Rehabilitation clients with DVR enrollment dates closest to their matched treatment group members. The other comparison group—referred to as the “historical” comparison group—had DVR enrollment dates that were approximately 2.5 years earlier than those of the treatment group. Each treatment group member was matched to one person from each comparison group pool. The matching variables included (1) experience as a DVR client, (2) experience as a Social Security disability benefit recipient while a DVR client, (3) primary disabling condition according to the DVR, (4) start date of DVR services, and (5) length of time since becoming eligible and starting DVR services. For both studies, the authors estimated impacts using mixed-effects linear models.
The studies used data from two sources. Quarterly earnings data for study participants were from the Vermont Department of Employment and Training. All other data were from the DVR. The studies’ outcome variable was earnings from competitive employment.
Study Sites
- VR offices and community mental health centers throughout Vermont.
Findings
2006 study. Among participants with psychiatric disabilities, VWII participation was correlated with significant increases in mean earnings. Treatment group members increased their earnings by $1,256 per person per year relative to the contemporaneous comparison group and $1,272 per person per year relative to the historical comparison group.
2004 study. Among all study participants, VWII participation was linked to earnings increases. Treatment group members increased their earnings by $900 and $760 per person per year relative to the contemporaneous and historical comparison groups, respectively.
Considerations for Interpreting the Findings
The authors mention that an SSI waiver program, which provided additional work incentives to SSI recipients, started during the intervention period and was only available to people who received benefits counseling—that is, treatment group members. Thus, for all SSI recipients who participated in the program, the study measured the effect of benefits counseling plus the SSI waiver.
In addition, the earnings data were drawn from the Vermont unemployment insurance records, and therefore do not include income obtained from self-employment, out-of-state work, elected office, religious nonprofit organizations, charitable and educational organizations, farm work, and some railroad work. The authors estimate that about 11 percent of study participants received earnings from sources not reported in state unemployment insurance records. Hence, the analyses’ dependent variable may contain substantial measurement error. Finally, the ethnic homogeneity of Vermont's population may limit the ability to generalize the study’s results to other states.
Causal Evidence Rating
The studies receive low causal evidence ratings because they do not adequately demonstrate baseline equivalence between the treatment and comparison groups or control in the analysis for pre-intervention earnings levels. Specifically, it is not clear whether differences in pre-intervention earnings levels were statistically significant or adequately controlled for. Consequently, we are not confident that the estimated effects are attributable to the VWII.
Additional Sources
Tremblay, T., Smith, J., Xie, H., & Drake, R. (2004). The impact of specialized benefits counseling services on social security administration disability beneficiaries in Vermont. Journal of Rehabilitation, 70(2), 5-11.