Citation
Riccio, J., Dechausay, N., Miller, C., Nunez, S., Verma, N., & Yang E. (2013). Conditional cash transfers in New York City: The continuing story of the Opportunity NYC—Family Rewards demonstration. New York, NY: MDRC.
Highlights
- The study’s objective was to examine the impact of Family Rewards, an experimental privately funded conditional cash transfer program in New York City, on earnings, employment, public benefits receipt, and education.
- The study was based on a randomized controlled trial and estimated the effect of the Family Rewards program on low-income families. The authors used New York City and New York State administrative data to compare average outcomes between families offered access to the program and families excluded from the program, after adjusting for chance initial differences between the groups.
- The authors found that, on average, the Family Rewards program decreased the likelihood of ever being employed at an Unemployment Insurance (UI)-covered job in the first year by 2 percentage points, but increased monthly earnings by $353 and decreased the share of families receiving income from Supplemental Security Income (SSI) or disability benefits by 3.9 percentage points. The authors also found that parents in the Family Rewards program group were 4.2 percentage points more likely to have achieved any trade license training certification.
- The quality of causal evidence presented in this report is high for most outcomes because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Family Rewards program, and not to other factors. However, some outcomes in the study receive a moderate or low causal evidence rating, meaning that we are less confident that the estimated effects are attributable to the Family Rewards program; other factors may have contributed.
Intervention Examined
The Family Rewards Program
Features of the Intervention
Family Rewards was a conditional cash transfer program that tied cash rewards to prespecified activities and outcomes in children’s education, families’ preventive health care, and parents’ employment. The program was created by private foundations and community-based organizations in New York City to test this type of program’s effect in a developed-country context. Cash assistance was offered to reduce immediate hardship and to build human capital to reduce poverty over the long-term. Participants could receive cash incentives if they met required conditions, including education-focused conditions such as school attendance, achievement levels on standardized tests, and engagement with students’ education; health-focused conditions such as maintaining health insurance coverage for parents and their children; and workforce-focused conditions aimed at parents, such as maintaining full-time work and participating in approved education and job-training activities. The program was scheduled to last three years.
Features of the Study
From July 2007 to January 2008, a total of 4,790 families were randomly assigned to a group that could participate in the Family Rewards program or to a control group that was not offered the incentives. Families eligible for the study were living in selected New York City community districts; had incomes at or below 130 percent of the federal poverty level; had at least one child in the 4th, 7th, or 9th grade; and composed only of legal residents of the United States. The authors compared outcomes between these two groups after adjusting for chance differences in the characteristics of the groups.
Findings
- The study found that, on average, the Family Rewards program decreased the likelihood of ever being employed at a UI-covered job in the first year by 2.4 percentage points.
- The study also found that, on average, the Family Rewards program increased household income in the month before the 42-month survey by $353 and decreased the share of families who received income from SSI or disability benefits by 3.9 percentage points.
- Access to the Family Rewards program was associated with higher likelihoods of achieving any trade license or training certification by 4.2 percentage points. (This outcome received a moderate evidence rating because, although many participants were missing data on this outcome, the authors controlled statistically for all the variables that the CLEAR protocol requires for a moderate rating.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to employment, income, public benefit receipt, and education. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Outcomes other than those reported above under the Findings section received a low evidence rating because enough participants were missing data on the outcomes that differences between the treatment and comparison group members who responded might have been driven by factors that determined response, not the Family Rewards program.The authors did not control statistically for all the variables that the CLEAR protocol requires for a moderate rating. This review was conducted in collaboration with the Employment Strategies for Low-Income Adults Evidence Review (ESER). Because ESER did not report findings for studies that received a low causal evidence rating, the CLEAR profile does not report the otherwise eligible, low-rating findings either.
Causal Evidence Rating
The quality of causal evidence presented in this report is high for most outcomes because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects on those outcomes are attributable to the Family Rewards program, and not to other factors. However, some outcomes in the study, including receipt of training certification, receive a moderate causal evidence rating, meaning that we are somewhat confident that the estimated effects are attributable to the Family Rewards program; other factors might also have contributed. Finally, an additional outcome, current employment from the 42-month survey, received a low causal evidence rating, which means we are not confident that the estimated effects are attributable to the Family Rewards program; other factors are likely to have contributed.