Citation
Dean, D., Dolan, R., & Schmidt, R. (1999). Evaluating the Vocational Rehabilitation Program using longitudinal data: Evidence for a quasi-experimental research design. Evaluation Review, 23(2), 162–189
Highlights
- The study’s objective was to determine the impact of participation in Vocational Rehabilitation (VR), a state and federal program, on earnings for people with disabilities. VR provides services and supports to people with disabilities to help them achieve their employment goals.
- The authors analyzed linked Rehabilitation Services Administration (RSA) and Social Security Administration (SSA) data for about 29,000 people whose VR cases were closed in 1980. The linked data contained annual earnings histories, service expenditures, and disability and demographic information.
- By the eighth year after first receiving VR services, women in six of seven disability categories and men in four of seven disability categories earned significantly more than their counterparts who had not received VR services.
- The quality of the causal evidence presented in this study is moderate. This means we have some confidence that the observed effects represent the impact of VR, although other factors might also have contributed.
Intervention Examined
The Vocational Rehabilitation (VOC) Programs
Features of the Intervention
The VR program, which is administered jointly by states and the federal government, provides services and supports to transition-age youth and working-age adults with disabilities to help them achieve their employment goals. These services and supports can include counseling, job training or placement, formal education, income support, and physical or mental health services. VR counselors provide guidance and job placement services but typically refer recipients to private organizations, such as educational institutions or health care companies, for other services.
Features of the Study
The authors analyzed data from the 1980 RSA-SSA DataLink, which contained annual earnings histories, service expenditures, disability type, and demographic information for a random 10 percent sample of VR clients whose cases were closed in 1980—a total of 28,986 observations. Members of the treatment group had received VR services, whereas those in the comparison group had applied and qualified for VR but never received VR services.
To estimate the impact of receiving VR services on earnings, the authors compared the differences in earnings of treatment and comparison group members from two years before to each of the first eight years after being found eligible for VR. The authors estimated VR impacts separately for women and men in each of seven disability categories: visual impairments, hearing and speech impairments, musculoskeletal conditions, internal conditions, mental illness, substance abuse, and intellectual disabilities.
Findings
- The study found that receiving VR services led to increased earnings for women in almost all disability categories in most post-program years.
- Among women, earnings gains in the short term were largest for those with mental illnesses—a gain of $1,450 per year more than comparison group members in the first post-program year—but this declined in magnitude over subsequent years. Women with visual impairments had increasing earnings gains over the period examined, from $1,090 in the first post-program year to $2,185 in the eighth post-program year (all in 1980 dollars).
- For men, receiving VR services led to increased earnings in some years in all disability categories except hearing and speech impairments. Men with musculoskeletal and internal disabilities saw consistent earnings gains relative to comparison group counterparts, in all post-program years, up to $1,870 (musculoskeletal) and $1,955 (internal) by the eighth post-program year. Men with substance abuse problems and intellectual disabilities realized gains in the first one to three post-program years, but these impacts faded thereafter.
Considerations for Interpreting the Findings
The study was a carefully executed fixed-effects design and therefore receives a moderate causal evidence rating, the highest available to nonexperimental designs. However, because the data did not contain information on changes in functional limitations over time, the results could be biased if people who received VR services experienced more changes in functioning over time than comparison group members did.
In addition, the study excluded all people whose earnings were truncated because they exceeded the maximum of the Federal Insurance Contribution Act tax base (about 2.7 percent of cases). This could artificially shrink the observed impacts because these participants’ earnings were high relative to the rest of the sample.
Causal Evidence Rating
The quality of causal evidence provided in this study is moderate because it is a well-implemented nonexperimental study. This means we have some confidence that the estimated effects reflect the impact of VR services on earnings, although other factors might also have contributed to the observed effects.