Citation
Saez, E. (2009). Details matter: The impact of presentation and information on the take-up of financial incentives for retirement saving. American Economic Journal: Economic Policy, 1(1), 204-228.
Highlights
-
- The study’s objective was to examine the impact of contribution matches, credit rebates, and advance notification on tax filers’ decisions about opening an individual retirement account (IRA) during the tax preparation process and the amount they contributed to the IRA.
- The authors randomly assigned tax filers at 60 H&R Block locations in St. Louis, Missouri, to treatment conditions, defined by whether the filers were offered a 50 percent one-time match on IRA contributions, a 33 percent credit rebate on IRA contributions, or a 50 percent match on one-time and monthly IRA contributions. H&R Block provided tax filing information from the 2005 and 2006 tax years as well as background information on the filers.
- The study found that offering a 50 percent match on one-time IRA contributions and offering a 33 percent credit rebate increased the likelihood of opening an IRA and the amount contributed, but the effect on the likelihood of opening an IRA was larger for the 50 percent match treatment group.
- The quality of causal evidence is high for some outcomes because they were based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the treatment under study, and not to other factors. However, the quality of causal evidence for other outcomes is low because the analyses were based on a nonrandom subset of the randomized sample, and the author did not use sufficient controls when estimating impacts.
Features of the Study
The author designed a randomized controlled trial in collaboration with 60 H&R Block locations in St. Louis, Missouri. Individuals who had filed their taxes at one of these locations in 2005 and returned in 2006 were eligible to participate. Members of the study sample were randomly assigned to one of several conditions, depending on the location at which they filed their taxes. The author constructed four study scenarios, in which all sample members, regardless of treatment condition, received a waiver for the IRA set-up fee:
-
- In 19 offices (15,852 individuals), 10 percent of filers were offered a 50 percent match on one-time IRA contributions and 10 percent were offered a 33 percent credit rebate on an IRA contribution. Those who took up the credit rebate received 33 percent of their contribution amount as cash back. In these offices, 80 percent of filers were assigned to the control group.
- In 20 offices (17,598 individuals), 10.4 percent of filers were offered a 50 percent match on one-time IRA contributions and received advanced notification of the match and fee waiver offer; 24.1 percent were offered a 50 percent match on one-time IRA contributions but did not receive advanced notification; 10.2 percent were assigned to the no-match group but did receive advanced notice of the IRA fee waiver; and 55.3 percent of filers were assigned to a control group that received no match and no advanced notification.
- In 21 offices (14,878 individuals), 11 percent of filers were offered a 50 percent match on both one-time and monthly IRA contributions. The remaining 89 percent of filers were assigned to the control group.
Tax filers were randomly assigned to study groups using the last two digits of their Social Security numbers; both the tax filer and the tax preparer were blind to treatment assignment. The author estimated impacts on the IRA take-up rate and contribution amounts using H&R Block tax filing records.
Findings
The author estimated impacts for several contrasts:
-
- Fifty percent match on one-time IRA contribution versus no match or credit control group: The average take up of an IRA was 5.72 percentage points higher and the average amount contributed was $50 greater in the 50 percent match group than the control group. Both impacts were statistically significant.
- Thirty-three percent credit rebate on one-time IRA contribution versus no match or credit control group: The average take-up of an IRA was 2.4 percentage points higher and the average amount contributed was $19.85 greater in the 33 percent credit rebate group than the control group. Both impacts were statistically significant.
- Fifty percent match versus 33 percent credit rebate on one-time IRA contributions: The offer of a 50 percent match increased the likelihood of making IRA contributions by a statistically significant 3.68 percentage points relative to the offer of a 33 percent credit rebate. However, there was no difference in the contribution amount.
- Fifty percent match versus no match or credit control group on one-time and monthly IRA contributions: The match offer increased the likelihood of opening a one-time and systematic IRA by a statistically significant 1.44 percentage points relative to no match or credit offered.
Considerations for Interpreting the Findings
The author randomly assigned all sample members to each treatment condition. However, in the treatment group that received advance notification, the author restricted the analytic sample to those who actually appeared at H&R Block to file their taxes in 2006. Because this is a nonrandom subset of those assigned to the advance notification group, the analyses involving this group must meet CLEAR evidence guidelines for nonexperimental studies, which require controls for age, gender, and income level. The author did not include controls for age or gender, so this analysis receives a low causal evidence rating.
The author noted that, because of tax preparer discretion, people in the study might not have received the treatment to which they were assigned. The author analyzed sample members based on the assigned treatment rather than the received treatment, so the effects presented in the study might underestimate the interventions’ impacts on those who were actually treated.
Causal Evidence Rating
The quality of causal evidence is high for some outcomes because they were based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the treatment under study, and not to other factors. However, the quality of causal evidence for other outcomes is low because the analyses were based on a nonrandom subset of the randomized sample, and the author did not use sufficient controls when estimating impacts.