Absence of conflict of interest.
Citation
Bos, J. M., Scrivener, S., Snipes, J., Hamilton, G., Schwartz, C., & Walter, J. (2002). Improving basic skills: The effects of adult education in welfare-to-work programs. New York: Manpower Demonstration Research Corporation.
Highlights
- The study’s objective was to examine the impact of both education- and employment-focused welfare-to-work programs on earnings and public benefit receipt.
- The study was a randomized controlled trial conducted in seven geographic areas. The authors randomly assigned welfare recipients to the treatment condition (an education-focused intervention or an employment-focused intervention) or the control group. The study sample included a subset of welfare recipients who lacked a high school diploma or GED.
- The study found that participation in the welfare-to-work programs significantly increased earnings and reduced public benefit receipt and that participation in the employment-focused programs had a greater impact on earnings.
- The quality of causal evidence presented in this report is low because it was based on a randomized controlled trial with unknown attrition, and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the effects are attributable the education-focused or employment-focused interventions; other factors are likely to have contributed.
Intervention Examined
The Education-Focused and Employment-Focused Interventions
Features of the Intervention
The National Evaluation of Welfare-to-Work Strategies (NEWWS) featured eleven welfare-to-work programs implemented in seven geographic locations in the United States: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon. NEWWS employed both education- and employment-focused interventions designed to move welfare participants to the workforce. Education-focused interventions emphasized activities to improve basic skills and spent a large amount of time on reading and writing activities (referred to as Human Capital Development). Employment-focused interventions emphasized activities that would help participants find employment quickly (referred to as Labor Force Attachment). The employment-focused approach was implemented across four sites (Atlanta, Grand Rapids, Riverside, and Portland), and the education-focused approach was implemented across six sites (Atlanta, Grand Rapids, Riverside, Columbus, Detroit, and Oklahoma City). Participants were required to engage in the activities of the intervention, or they would receive sanctions that reduced their amount of public assistance.
Features of the Study
The study used a randomized controlled trial to assess the impact of the NEWWS welfare-to-work programs on outcomes. In Atlanta, Grand Rapids, and Riverside, welfare recipients were randomly assigned to one of three conditions: an education-focused intervention, an employment-focused intervention, or the control group. In Columbus, welfare recipients were randomly assigned to a traditional education-focused intervention, an education-focused intervention featuring integrated case management, or the control group. Welfare recipients in Detroit and Oklahoma City were randomly assigned to either the education-focused intervention or control group, whereas welfare recipients in Portland were randomly assigned to either the employment-focused intervention or control group.
The study sample included a subset of 19,589 welfare recipients served by the NEWWS programs focusing on individuals who lacked a high school diploma or GED. The sample was composed of low-income, mostly unemployed, single-parent women. Participants assigned to the treatment group had access to education, training, and other employment services and were required to participate or risk a reduction in their monthly welfare grant. The control group participants received no services, but they could seek out services in the community. Data were collected from June 1991 through December 1997 using multiple sources: baseline enrollment forms, surveys, baseline achievement tests, and administrative data. After controlling for several factors, the authors used statistical models to systematically compare the program effects across all 11 programs and seven sites to determine the most effective approach. Estimates represent the differences between the participation levels of the control and program groups.
Study Sites
- Atlanta, Georgia
- Columbus, Ohio
- Detroit, Michigan
- Grand Rapids, Michigan
- Oklahoma City, Oklahoma
- Portland, Oregon
- Riverside, California
Findings
Earnings and wages
- The study found that participation in the education- or employment-focused programs was significantly associated with increased earnings, where program participants who did not hold a high school diploma or GED earned an average of $1,212 more than individuals in the control group.
- The study also found that participation in any welfare-to-work program was significantly related to third-year earnings, where program participant earnings increased by $543, representing a gain of 21.8 percent.
- However, the study found that employment-focused programs have a larger impact on earnings during the early program years.
Public benefits receipt
- The study found that participation in the education- or employment-focused programs was significantly associated with a reduction in Aid to Families with Dependent Children (AFDC) receipt, where program participants received $1,056 fewer AFDC benefits than individuals in the control group; a reduction of 10.6 percent.
- The study also found that participation in any welfare-to-work program was significantly related to third-year welfare benefits, where program participant welfare benefits decreased by $390 or 14.4 percent.
Considerations for Interpreting the Findings
The authors conducted a randomized controlled trial; however, they only provide enough information to calculate overall attrition (6.4%). Since no sample sizes were provided for each treatment and control group, differential attrition is unknown. CLEAR guidelines require that both overall and differential attrition are reported. Additionally, although the authors controlled for several group differences at baseline and included a variety of control variables in the models, they did not control for previous earnings greater than one year before program participation as required by the protocol. These preexisting differences in earnings—and not the education-focused or employment-focused interventions—could explain the observed differences in outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because it was based on a randomized controlled trial with unknown attrition, and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the effects are attributable the education-focused or employment-focused interventions; other factors are likely to have contributed.