Skip to main content
UNITED STATES DEPARTMENT OF LABOR

The Indiana welfare reform evaluation: Five-year impacts, implementation, costs, and benefits (Beecroft et al. 2003)

Review Guidelines

Causal Evidence Guidelines

Absence of Conflict of Interest

This study was conducted by staff from Abt Associates, which administers CLEAR. The review of this study was conducted by ICF Incorporated, which also administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.

Citation

Beecroft, E., Lee, W., Long, D., Holcomb, P. A., Thompson, T. S., Pindus, N., O'Brien, C., & Bernstein, J. (2003). The Indiana welfare reform evaluation: Five-year impacts, implementation, costs, and benefits. Report to the Indiana Family and Social Services Administration, Division of Family and Children. Cambridge, MA: Abt Associates. [Early cohort single parent households]

Highlights

  • The study's objective was to examine the impact of the Indiana Welfare Reform on employment, earnings, and public benefits receipt outcomes. This profile focuses on the outcomes for the single parent households in the early cohort. The authors investigated similar research questions for another contrast, the profile of which can be found here.
  • The study was a randomized controlled trial that used administrative records, quarterly earnings data, and participant surveys to compare the outcomes of the treatment and control group members.
  • The study found that Indiana's program significantly reduced public benefits receipt and increased earnings and employment rates over the five-year study period.
  • This study receives a high evidence rating. This means we are confident that the estimated effects are attributable to Indiana’s Welfare Reform, and not to other factors.

Intervention Examined

Indiana’s Welfare Reform

Features of the Intervention

Indiana's welfare program is designed for disadvantaged single-parent families needing financial assistance. Due to welfare reform at the federal level, Indiana's welfare program underwent policy changes between 1997 and 2000 to strengthen the program. Indiana’s Family and Social Services Administration (FSSA) focused on employment and training, including unsubsidized employment placement and job search activities. Program participants were required to participate in work activities for at least 20 hours per week; this was raised to 25 hours per week in June 1997. They were also required to sign a personal responsibility agreement (PRA), ensuring that their children were immunized and that school-age children attended school. Participants could receive sanctions (a reduction in the amount of cash assistance received) for noncompliance with the work requirement or PRA. However, participants were exempt from the work requirements if caring for a child under age 3. The exemption age was lowered to age 2 in June 1997 and then to age 1 in December 1997. Temporary Assistance to Needy Families (TANF) receipt was limited two years and the amount would not increase if a child was conceived while the mother was receiving TANF benefits. FSSA also increased its earnings disregard to provide a stronger work incentive and make it easier to increase income. The fixed grant policy for the most “job ready” clients (Placement track clients) did not change the TANF amount received with increased earnings. There was also a zero grant policy where participants would remain eligible for Medicaid and other supportive services if there was an increase in earnings.

Features of the Study

The study was a randomized controlled trial where eligible families that were approved for welfare and cash assistance were randomly assigned to the Welfare Reform group (treatment) or the Traditional Welfare group (control). The treatment group received the new welfare reform policies, and the control group received the traditional policies that also included their participation in the work component of the Indiana welfare program. Single-parent families from all 92 counties across the state of Indiana that received welfare between May 1995 and April 1996 were eligible for the study. The study sample included 64,440 families (63,223 in the treatment group and 3,217 in the control group). The majority of participants were female (97 percent), White (58 percent) or African American (38 percent), with an average age of 29 years. The study data included administrative records from FSSA's TANF eligibility computer system, quarterly earnings records from the State Unemployment Insurance System, and a five-year follow-up survey. The authors used statistical models to compare the outcomes of treatment and control group members at various time points after program entry.

Findings

Earnings and wages

  • The study found that Indiana's welfare reform program significantly increased earnings for participants in each of the follow-up years. Over the five-year period, average earnings were $2,096 higher for the treatment participants compared to control participants. This finding was significant.

Employment

  • The study found that Indiana's welfare reform program significantly increased employment rates for participants in each of the follow-up years, with treatment participants having a higher average quarterly employment rate over the study period compared to control participants (52 percent vs. 49 percent, respectively).
  • The study found that more treatment participants than control participants were ever employed over the study period (89 percent vs. 87 percent), and were employed and off TANF and food stamps in each year except year 4. These differences were significant.

Public benefits receipt

  • Over the five-year follow-up period, the study found that significantly fewer treatment participants than control participants were receiving TANF payments in the last quarter of each year.
  • The study found that treatment participants received TANF for fewer months over the study period than control participants, and that treatment participants received $1,027 less in average TANF payments over the study period. These differences were significant.
  • The study found that Indiana's program also reduced food stamp receipt, with significantly fewer treatment participants than control participants receiving food stamps in the last quarter of each year except year 4.  
  • The study found that treatment participants received food stamps for fewer months over the study period than control participants, and that treatment participants received $420 less in average food stamps over the study period. These differences were significant.

Considerations for Interpreting the Findings

The study authors estimated multiple related impacts on outcomes related to employment, earnings, and public benefit receipt. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated. Also, the study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.

Causal Evidence Rating

The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to Indiana’s Welfare Program, and not to other factors.

Reviewed by CLEAR

July 2022

Topic Area