Skip to main content
UNITED STATES DEPARTMENT OF LABOR

Evaluating two approaches to case management: Implementation, participation patterns, costs, and three-year impacts of the Columbus Welfare-to-Work Program (Scrivener & Walter 2001)

Review Guidelines

Causal Evidence Guidelines

Absence of conflict of interest.

Citation

Scrivener, S, & Walter, J. (2001). Evaluating Two Approaches to Case Management: Implementation, Participation Patterns, Costs, and Three-Year Impacts of the Columbus Welfare-to-Work Program. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of the Assistant Secretary for Planning and Evaluation and U.S. Department of Education. [Integrated Case Management vs. Control]

Highlights

  • The study's objective was to examine the impact of an integrated case management approach on employment, earnings, and public benefits receipt. This profile focuses on the comparison of participants in the integrated program group versus participants in the control group. The authors investigated similar research questions for other contrasts, the profiles of which can be found here.
  • The study was a randomized controlled trial that assigned welfare applicants to the integrated case management, traditional case management, or control groups. Using state administrative records data, the authors conducted statistical models to compare the outcomes of participants in the integrated program and control groups in the third year after random assignment.
  • The study found that integrated program participants were significantly more likely to be employed, have higher earnings, and less likely to receive public benefits compared to control group participants.
  • This study receives a high evidence rating. This means we are confident that the estimated effects are attributable to the Integrated Case Management program, and not to other factors.

Intervention Examined

Integrated Case Management

Features of the Intervention

To help improve educational and vocational skills for single-parent Aid to Families with Dependent Children (AFDC) applicants and recipients, Integrated Case Management provides personalized case management to help individuals prepare for and secure a job. Participants in the integrated program had one case manager who did all the required public benefit and case management duties (e.g., determine welfare eligibility and payment issuance). After determining participants were employable, case managers provided job search assistance and supportive services. Supportive services included childcare, transportation, and other incidental work costs, and the program had an on-site childcare center. The participants who did not have a high school diploma or general education diploma were assigned to basic education classes. The participants with basic education credentials were assigned to vocational training, postsecondary education, or work experience. Program participation was closely monitored and enforced by case managers and sanctions were imposed on participants that did not meet the program participation requirements by reducing monthly welfare payments. The program serves welfare recipients whose youngest child was at least 3 years old and did not meet any of the federal exemption criteria for participating in the program.

Features of the Study

The study was a randomized controlled trial in Columbus, Ohio. Applicants who enrolled in the study between September 1992 and July 1994 were randomized to one of two treatment groups (Integrated Case Management or Traditional Case Management) or a control group. Of the 7,242 applicants, the authors assigned 2,513 applicants to the Integrated Case Management treatment group to receive program services, which included employment and training services. The treatment group was required to participate in program activities or risk a reduction in monthly AFDC dollars. The authors assigned 2,159 applicants to the control group that did not have access to program services but could independently pursue similar services in the community. The study sample was primarily female (94 percent), with an average age of 32 years. Over half of the sample was Black (52 percent) and had at least a high school diploma or GED (57 percent). Employment, earnings, and public benefits receipt outcomes were examined using state administrative records data for up to three years after random assignment. The authors used a statistical model to compare the outcomes of participants in the integrated program and control groups.

Findings

Earnings and wages

  • The study found that integrated program participants earned $1,181 more than control participants across the three study years, earned $596 more in year two and $505 more in year 3. These differences were significant. However, the study found no significant differences in earnings in year one.

Employment

  • The study found that integrated program participants were significantly more likely than control participants to be employed across the study period (81 percent vs. 79 percent) and in year 3 (69 percent vs. 65 percent). However, the study found no significant differences in employment during years one and two.
  • The study also found that integrated program participants were employed in more quarters across the study period and in years two and three than control participants. These findings were significant. No significant differences were found for year one.

Public benefits receipt

  • The study found that integrated program participants received AFDC for fewer months over the study period than control participants, and that integrated program participants received $1,079 less in AFDC benefits over the study period. These differences were significant.
  • The study found that significantly fewer integrated program participants than control participants were receiving AFDC in year two (65 percent vs. 69 percent) and year three (47 percent vs. 54 percent). No significant differences were found for year one.

Considerations for Interpreting the Findings

The study authors estimated multiple related impacts on outcomes related to earnings and wages, employment, and public benefits. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated. Additionally, the study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.

Causal Evidence Rating

The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Integrated Case Management program, and not to other factors.

Reviewed by CLEAR

August 2022

Topic Area