Absence of conflict of interest.
Citation
Leininger, L. J., & Kalil, A. (2008). Cognitive and non-cognitive predictors of success in adult education programs: Evidence from experimental data with low-income welfare recipients. Journal of Policy Analysis and Management, 27, 521-535. doi: 10.1002/pam.20357
Highlights
- The study’s objective was to examine the impact of an education-focused intervention and an employment-focused intervention on education outcomes.
- The authors used data from a randomized controlled trial, the National Evaluation of Welfare-to-Work Strategies (NEWWS). The NEWWS study randomly assigned individuals within seven geographic locations to one of three conditions: an education-focused intervention, an employment-focused intervention, or the control group. The current study used a subsample to examine participants’ completion of a high school diploma or equivalent two years after being assigned to one of the conditions.
- The study found that participation in the education-focused intervention was significantly associated with a higher likelihood of receiving a high school diploma or equivalent relative to the control group.
- The quality of the causal evidence presented in this study is moderate because, although it was based on a randomized controlled trial with unknown attrition, the authors demonstrated that the treatment and control groups were similar before the intervention. This means we have confidence that the estimated effects are attributable at least in part to the education-focused or employment-focused interventions, although other factors could also have contributed.
Intervention Examined
The Education-Focused and Employment-Focused Interventions
Features of the Intervention
The National Evaluation of Welfare-to-Work Strategies (NEWWS) was conducted in seven geographic locations in the United States: Atlanta, Georgia; Grand Rapids, Michigan; Riverside, California; Columbus, Ohio; Detroit, Michigan; Oklahoma City, Oklahoma; and Portland, Oregon. NEWWS implemented two interventions designed to move welfare participants to the workforce. Education-focused interventions emphasized activities to improve basic skills and spent a large amount of time on reading and writing activities (referred to as Human Capital Development). Employment-focused interventions emphasized activities that would help participants find employment quickly (referred to as Labor Force Attachment). Participants were required to participate in the activities of the intervention or they would receive sanctions that reduced their amount of public assistance.
Features of the Study
The study used data from the NEWWS – a randomized controlled trial where welfare recipients were randomly assigned to the education-focused intervention, employment-focused intervention, or control group. The current study includes data from three of the seven sites: Atlanta, GA; Grand Rapids, MI; and Riverside, CA. The authors analyzed data for a subsample of NEWWS participants who completed the two-year follow-up surveys. The participants included in the analysis were limited to those who: (1) did not have a high school diploma or GED at the baseline assessment; (2) completed the Personal Opinion Survey at baseline; and (3) completed numeracy and literacy tests at baseline. This resulted in a total of 2,076 participants (734 in the education-focused group, 513 in the employment-focused group, and 829 in the control group). The participants in each group were also stratified on non-cognitive (depression and locus of control) and cognitive (literacy and numeracy) factors. The authors conducted statistical models to examine differences between the groups on earning a high school diploma or GED two years after being assigned to one of the conditions.
Findings
Education and skills gains
- The study found that participation in the education-focused intervention was significantly associated with a higher likelihood of earning a high school diploma or GED than individuals in the control condition (an 8.3 percentage point difference).
- The study also found that participation in the education-focused intervention for those with high literacy skills was significantly associated with a higher likelihood of earning a high school diploma or GED than individuals in the control condition (a 14.8 percentage point difference).
- The study did not find a significant association between participation in the employment-focused intervention and earning a high school diploma or GED.
Considerations for Interpreting the Findings
The study used data from a randomized controlled trial. The authors note that the response rate for the survey two years after baseline was 75 percent in Riverside and 90 percent in Atlanta and Grand Rapids with “similar response rates” across treatment and control groups. The data across the three sites was pooled for analysis; therefore, the actual attrition is unknown since no sample sizes were provided for each group at the individual sites. However, the authors controlled for group differences at baseline and included a variety of control variables in the models such as age, race/ethnicity, gender, and being on welfare.
Causal Evidence Rating
The quality of the causal evidence presented in this report is moderate because, although it was based on a randomized controlled trial with unknown attrition, the authors demonstrated that the treatment and control groups were similar before the intervention. This means we have confidence that the estimated effects are attributable at least in part to the education-focused or employment-focused interventions, although other factors could also have contributed.
Additional Sources
Hamilton, G., Freedman, S., Gennetian, L., Michalopoulos, C., Walter, J., Adams-Ciardullo, D., Gassman-Pines, A., McGroder, S., Zaslow, M., Brooks, J., Ahluwalia, S., Small, E., & Ricchetti, B. (2001). National evaluation of welfare-to-work strategies: How effective are different welfare-to-work approaches? Five-year adult and child impacts for eleven programs. New York, NY: Manpower Demonstration Research Corporation.