Title,Citation,Topic_area,Study_type,Study_evidence_rating,Outcome_effectiveness,Findings,Intervention_program,Topics,Target_population,Firm_characteristics,Geographic_setting,Original_publication_date,Original_publication_link,"Review Protocol" "Small firms, bigger changes: Health insurance coverage take-up rates in small firms after the ACA","Kattih, N., Mansour, F., & Mixon, F. G. (2019). Small firms, bigger changes: Health insurance coverage take-up rates in small firms after the ACA. Applied Economics, 51(54), 5878–5889. doi:10.1080/00036846.2019.1630710",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Unfavorable impacts Employer benefits receipt","Summary: The study’s objective was to examine the impact of the Affordable Care Act of 2010 (ACA) on employee health insurance take-up rates.  The study was a nonexperimental analysis that used a statistical model to examine the impact of the ACA on employee health insurance take-up rates in small firms in the United States. The study used 2006–2017 data from the Kaiser Family Foundation and Health Research Educational Trust Survey of Employer Health Benefits. The authors compared employee health insurance take-up in small firms with fewer than 50 employees to larger firms as a comparison group.  The study found a statistically significant decrease in employee health insurance coverage take-up for small firms after the implementation of the ACA.   The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the ACA; other factors are likely to have contributed. ","Affordable Care Act (2010)","Health insurance",Adult,,,2019,https://www.tandfonline.com/doi/abs/10.1080/00036846.2019.1630710?journalCode=r…,"Review Protocol" "Conditional cash transfers in New York City: The continuing story of the Opportunity NYC—Family Rewards demonstration","Riccio, J., Dechausay, N., Miller, C., Nunez, S., Verma, N., & Yang E. (2013). Conditional cash transfers in New York City: The continuing story of the Opportunity NYC—Family Rewards demonstration. New York, NY: MDRC.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of Family Rewards, an experimental privately funded conditional cash transfer program in New York City, on earnings, employment, public benefits receipt, and education. The study was based on a randomized controlled trial and estimated the effect of the Family Rewards program on low-income families. The authors used New York City and New York State administrative data to compare average outcomes between families offered access to the program and families excluded from the program, after adjusting for chance initial differences between the groups. The authors found that, on average, the Family Rewards program decreased the likelihood of ever being employed at an Unemployment Insurance (UI)-covered job in the first year by 2 percentage points, but increased monthly earnings by $353 and decreased the share of families receiving income from Supplemental Security Income (SSI) or disability benefits by 3.9 percentage points. The authors also found that parents in the Family Rewards program group were 4.2 percentage points more likely to have achieved any trade license training certification. The quality of causal evidence presented in this report is high for most outcomes because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Family Rewards program, and not to other factors. However, some outcomes in the study receive a moderate or low causal evidence rating, meaning that we are less confident that the estimated effects are attributable to the Family Rewards program; other factors may have contributed.","the Family Rewards Program","Other employment and reemployment Other training and education","Low income, Parent",,"United States, Urban",2013,http://files.eric.ed.gov/fulltext/ED545453.pdf,"Employment and Training Review Protocol" "Investing for retirement: The moderating effect of fund assortment size on the 1/n heuristic","Morrin, M., Inman, J. J., Broniarczyk, S. M., Nenkov, G., & Reuter, J. (2012). Investing for retirement: The moderating effect of fund assortment size on the 1/n heuristic. Fox School of Business Research Paper No. 14-009, 1–38.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: Experiments using U.S. households The study’s objective was to examine whether increases in the number of funds offered in a retirement plan caused investors to allocate their contributions more evenly across all available investment options. The authors administered one survey to a nationally representative sample of U.S. households and two surveys to people from consumer panels. The surveys contained information on hypothetical savings plans that were manipulated to include either a small or large fund assortment size. Participants were instructed to choose the plans in which they would invest and indicate the percentage they would invest in each of the chosen funds. The study found that participants were more likely to allocate hypothetical investments across all available funds when offered a small versus large assortment size. The quality of the causal evidence presented in this portion of the study is moderate because, although the authors controlled for observable characteristics of samples members and there was no evidence that survey versions were systematically assigned to sample members, the Clearinghouse for Labor Evaluation and Research (CLEAR) could not confirm that survey versions were randomly assigned. Analysis of employees’ data  The study’s objective was to determine whether the number of fund options offered by a 401(k) plan influenced the investment behavior of plan members. The authors compared investment behaviors among employees offered 10 or 19 fund options for investment, controlling for employees’ characteristics. The analysis used administrative data on contributions to the defined contribution plan of the Oregon University System. The study used an interrupted time series (ITS) design that analyzed changes in outcomes before and after an increase in the number of available investment options. The analysis demonstrated that offering a larger fund assortment size was associated with a statistically insignificant decrease in the tendency of employees to invest in all available funds and spread invested dollars evenly among chosen alternatives. The quality of causal evidence presented in this study is low. This means that we are not confident that the estimated effects are solely attributable to fund assortment size; other factors are likely to have contributed.","Informational Interventions for Households and Workers","Employer provided retirement benefits",Employed,,"United States",2012,,"Behavioral Finance: Retirement Review Protocol" "What will my account really be worth? An experiment on exponential growth bias and retirement saving","Goda, G.S., Manchester, C.F., & Sojourner, A. (2012). What will my account really be worth? An experiment on exponential growth bias and retirement saving. National Bureau of Economic Research working paper 17927. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine how general or personalized information on projected retirement savings account balances and annual retirement income affects employees’ retirement savings decisions. A group of university employees were randomly assigned to one of four conditions: Participants in the planning treatment group received general retirement savings information, including enrollment instructions for the university’s voluntary retirement plan. Participants in the balance treatment group received all planning treatment materials plus personalized projections of retirement fund balances and access to an online customization tool. Participants in the income treatment group received all planning and balance treatment materials plus a personalized projection of income during retirement. Participants in the control group received no intervention and were in departments with only control group members. The study analyzed retirement-fund contribution data from the university’s Office of Human Resources using fixed-effects regressions. The study also collected and analyzed additional survey data regarding financial literacy, tolerance for delayed gratification, tendency to procrastinate, and beliefs about saving. Participants in the income treatment group contributed $85.42 more on average, annually, to their retirement savings accounts than did members of the control group. Both this group and those in the balance treatment group exhibited statistically significant increases in the probability of changing their contribution amounts of 1.2 and 1.4 percentage points, respectively. The quality of causal evidence provided in this study is high. This means that we are confident that the estimated effects are attributable to the different levels of the intervention and not to other factors.","Providing General Information and Personalized Savings and Income Projections","Employer provided retirement benefits",Employed,,"United States",2012,http://www.nber.org/papers/w17927,"Behavioral Finance: Retirement Review Protocol" "Small cues change savings choices","Choi, J., Haisley, E., Kurkoski, J., & Massey, C. (2012). Small cues change savings choices. National Bureau of Economic Research working paper 17843. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to gauge the effect of savings cues on 401(k) contribution rates by exposing randomly selected groups of employees to different hypothetical savings rates or amounts. The authors analyzed the relationship between exposure to savings rate cues and savings choices among employees at an unnamed U.S. technology company who contributed less than the maximum yearly amount to their retirement savings plans prior to the intervention. Overall, the authors found that participants adjusted their 401(k) contribution rates in response to cues. Those who were cued with the maximum contribution amount were 5.7 percent more likely than those who received no cue to contribute at maximal levels. The quality of causal evidence provided in this study is low because it is a randomized controlled trial with unknown attrition and unknown equivalence of the analytic groups. This means we are not confident the estimated effects are attributable to savings cues; other factors are likely to have contributed.","Savings Cue","Employer provided retirement benefits",Employed,,"United States",2012,http://www.nber.org/papers/w17843,"Behavioral Finance: Retirement Review Protocol" "Thrift Savings Plan. (2012). Participant behavior and demographics: Analysis of 2008–2012.","Thrift Savings Plan. (2012). Participant behavior and demographics: Analysis of 2008–2012.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Mixed impacts Employer benefits receipt","Summary: The study’s objective was to determine whether changes in the design of the Thrift Savings Plan (TSP) led to changes in federal employees’ participation in and contributions to the Federal Employee Retirement System (FERS). In 2009 and 2010, TSP instituted two major changes, a decrease in the waiting period for eligibility to receive employer matching contributions and automatic enrollment in a Government Securities Investment Fund (G Fund). This study used an interrupted time series design to compare employees’ investment behaviors before and after the TSP design changes. It used data from the TSP record-keeping system and the Office of Personnel Management (OPM). Across the TSP participants analyzed, participation in FERS increased by about 4 percentage points from 2009 to 2012, whereas the average deferral rate decreased slightly from the 2008 level. TSP’s design changes were not associated with how often employees monitored their FERS investments. The quality of causal evidence presented in this study is low, meaning that we are not confident that the estimated effects are attributable to TSP design changes. Other factors are likely to have contributed.","Immediate Contributions and Automatic Enrollment in the Thrift Savings Plan","Employer provided retirement benefits",Employed,,"United States",2012,,"Behavioral Finance: Retirement Review Protocol" "Boon or bane?: 401(k) loans and loan provisions","Wenger, J., & Weller, C. (2011). Boon or bane?: 401(k) loans and loan provisions. Available at http://ssrn.com/abstract=1941411.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to examine the impact of having a borrowing option in a defined contribution (DC) pension plan on contribution rates and total debt. The authors used data from the United States Federal Reserve’s triennial Survey of Consumer Finance to estimate ordinary least squares (OLS) and instrumental variable (IV) regression models. Both models control for demographic characteristics and financial situation. The authors found that having a borrowing option significantly increased both contribution rates and debt. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the borrowing option in DC pension plans. Other factors are likely to have contributed.","Borrowing Options in Defined Contribution Pension Plans","Employer provided retirement benefits",Employed,,"United States",2011,http://ssrn.com/abstract=1941411,"Behavioral Finance: Retirement Review Protocol" "$100 bills on the sidewalk: Suboptimal investment in 401(K) plans","Choi, J., Laibson, D., & Madrian, B. (2011). $100 bills on the sidewalk: Suboptimal investment in 401(K) plans. The Review of Economics and Statistics, 93(3), 748-763.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-No impacts Employer benefits receipt","Summary: The study examined the impact of providing information about employer contribution matches and penalty-free withdrawal rules on 401(k) contribution rates. These features created an opportunity for certain employees to increase their retirement savings without decreasing take-home pay. Even though this enabled employees to make a profit at very little cost, many eligible employees did not choose to exploit this opportunity. Employees were randomly assigned to receive a survey embedded with explicit information on the opportunity or a control survey without the information. The authors used survey data linked to administrative data obtained from Hewitt Associates, a large benefits administration and consulting firm. The study found no statistically significant relationship between the treatment condition and future 401(k) contribution rates. The quality of the causal evidence presented in this study is high. This means we are confident that any estimated effects of the intervention would be attributable to the intervention itself, and not some other factor.","the Provision of Information on a Missed ""Free Lunch""","Employer provided retirement benefits",Employed,,"United States",2011,,"Behavioral Finance: Retirement Review Protocol" "Simplification and saving","Beshears, J., Choi, J., Laibson, D., & Madrian, B. (2010). Simplification and saving. Cambridge, MA: National Bureau of Economic Research.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s key objective was to examine the impact of Quick Enrollment, an option enabling individuals to more quickly and easily enroll in a retirement savings plan, on plan enrollment rates and contribution levels at two firms in the United States (companies A and B). The study also examined the impact of Easy Escalation, an option enabling individuals to quickly increase their retirement-plan contribution levels to a predetermined percentage of pay, on contribution rates. The analysis was based on data on enrollment in and contributions to retirement plans from Hewitt Associates, a benefits administration and consulting firm. The study used an interrupted time series (ITS) design, analyzing changes in outcomes before and after the implementation of Quick Enrollment or Easy Escalation. The study found that Quick Enrollment increased enrollment rates by 10 to 20 percentage points and that Easy Escalation increased the percentage of employees contributing to their savings plans at the higher, predetermined rate specified by Easy Escalation.  The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to Quick Enrollment or Easy Escalation. Other factors are likely to have contributed to the changes in enrollment and contribution rates.","the Quick Enrollment and Easy Escalation","Employer provided retirement benefits",Employed,"Health care and social assistance, Manufacturing","United States",2010,,"Behavioral Finance: Retirement Review Protocol" "Optimal defaults and active decisions","Carroll, G., Choi, J., Laibson, D., Madrian, B., & Metrick, A. (2009). Optimal defaults and active decisions. The Quarterly Journal of Economics, 124(4).","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study examined the impact of requiring employees to actively decide whether to enroll in a 401(k) plan, called active-decision enrollment, on newly hired employees’ 401(k) enrollment decisions and contribution rates at a large U.S.-based company. Active-decision enrollment was compared with standard enrollment, which required employees to opt in to a 401(k) plan. The analysis was based on administrative data on enrollment in and contributions to a retirement plan. The study analyzed changes in outcomes before and after the change from active-decision to standard enrollment. The study found that requiring employees to actively opt in or out of savings plans increased 401(k) participation and contribution rates, compared with standard enrollment. The study also found that employees who participated in the 401(k) plan under the active-decision enrollment policy contributed less, on average, than participants under standard enrollment.  The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the company’s active-decision enrollment policies. Other factors are likely to have contributed.","Active-Decision 401(k) Enrollment","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "New ways to make people sSave: A social marketing approach","Lusardi, A., Keller, P. A, & Keller, A. M. (2009). New ways to make people sSave: A social marketing approach. National Bureau of Economic Research Working Paper 14715. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to assess the effect of a retirement planning aid on encouraging new hires at a large institution to enroll in a supplemental retirement account (SRA). The authors presented results from both an initial pilot test and a full-scale implementation using a modified version of the planning aid. The authors compared the mean 30- and 60-day enrollment rates of cohorts of new hires using administrative data provided by the institution. The final version of the planning aid increased SRA enrollment considerably relative to the control group, with more than 27 percent of new hires enrolling within 30 days and more than 41 percent enrolling within 60 days, compared with 7 and 29 percent, respectively, in the control group. Results were similar during the pilot intervention phase. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to the planning aid; other factors are likely to have contributed.","a Retirement Planning Tool","Employer provided retirement benefits",Employed,,"United States",2009,http://www.nber.org/papers/w14715,"Behavioral Finance: Retirement Review Protocol" "Reducing the complexity costs of 401(k) participation through Quick Enrollment","Laibson, D., Choi, J., & Madrian, B. (2009). Reducing the complexity costs of 401(k) participation through Quick Enrollment. Developments in the Economics of Aging, 57-82.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to examine the effect of a simplified 401(k) enrollment procedure called Quick Enrollment on plan participation through three trials at two anonymous companies (two trials at company A and one at company B). Quick Enrollment offered employees the choice of joining a 401(k) plan at a fixed contribution rate, investing in prespecified funds. Hewitt Associates, a consultancy and benefits management company, provided data on 401(k) plan participation, compensation, and other demographic characteristics for employees at two anonymous firms. For company A, year-end cross-sectional data from 2002 and 2003 were available for all employees in addition to a September 1, 2004, snapshot. For company B, only year-end data for 2002 and 2003 were available. Plan enrollment rates for new employees rose after each Quick Enrollment implementation, with an enrollment increase of 14 percentage points at company A and 2 percentage points at company B. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to Quick Enrollment; other factors are likely to have contributed.","Quick Enrollment","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "Mental accounting in portfolio choice: Evidence from a flypaper effect","Choi, J., Laibson, D., & Madrian, B. (2009). Mental accounting in portfolio choice: Evidence from a flypaper effect. American Economic Review, 99(5), 2085-2095.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Unfavorable impacts Employer benefits receipt","Summary: The study examined the impact of requiring employees at a large, U.S.-based company to actively choose the asset allocation for an employer’s matching 401(k) contributions, rather than automatically allocating such funds to the employer’s stock. The analysis was based on administrative data on employees’ demographic characteristics and 401(k) contributions. The study analyzed changes in outcomes that occurred when a firm began requiring employees to choose the funds to which the employer’s 401(k) matching contributions would be allocated.  The study found that allowing employees to choose the asset allocation of their matched contributions reduced contributions to employer stock by 64 percentage points in the first year of the new policy. The study also found that employees’ own contribution allocations did not change significantly after the policy was enacted, suggesting that employees made their personal allocation decisions without considering the employer’s match allocation. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the active-decision requirement for employer matched contributions. Other factors are likely to have contributed.","Active Portfolio Choice for Employer Matching Contributions","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "Effects of public policies on the disposition of pre-retirement lump-sum distributions: Rational and behavioral influences","Burman, L., Coe, N., Dworsky, N., & Gale, W. (2008). Effects of public policies on the disposition of pre-retirement lump-sum distributions: Rational and behavioral influences. CentER Discussion Paper Series No. 2008-94.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine whether tax policies that differently frame the rollover of lump-sum distributions (LSDs) from defined-contribution retirement plans into tax-preferred funds affected rollover behavior. The authors investigated two tax reforms, the first in 1986 and the second in 1993. The 1986 reform levied an additional tax of 10 percent on LSDs (framed as a penalty) that were not rolled over into tax-preferred funds. The 1993 reform required LSDs to provide an automatic rollover option and increased tax withholding rates on cash LSDs. The authors used data from the 1993 Current Population Survey Employee Benefit Supplement to analyze the effects of the 1986 tax reform and data from the 1992–2004 Health and Retirement Study to assess the effect of the 1993 tax reform. Treating the tax rate on LSDs as a penalty in the 1986 reform framework increased LSD rollovers by 16 to 32 percentage points for the average individual in the study. The 1993 reforms led to a 10 to 13 percentage point increase in rollovers for the average individual in the study. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to the intervention alone; other factors are likely to have contributed.","the 1986 Tax Reform Act and 1993 Addendum","Federal retirement benefits",Employed,,"United States",2008,,"Behavioral Finance: Retirement Review Protocol" "The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data.","Haider, S. J., & Loughran, D. S. (2008). The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data. Journal of Human Resources, 43(1), 57-87.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Mixed impacts Earnings and wages Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective is to examine the impact of 1983 and 2000 policy changes that removed the Social Security earnings test for certain age groups on the earnings and employment outcomes of affected men The study is a nonexperimental analysis that examines the correlations between the 1983 and 2000 policy changes and earnings and employment status. The study compares the outcomes of men affected by the policy changes with the outcomes of men of similar ages not affected by the policy changes. The study finds that the 1983 policy change is associated with few changes to the observed outcomes and that the 2000 policy change is associated with higher earnings and hours worked per week for affected men. The quality of causal evidence presented in this report is low because the study does not demonstrate that men affected by the policy changes are similar to the men not affected by the policy change, nor does it control for possible differences. This means we are not confident that the estimated effects are attributable to the 1983 and 2000 policy changes; other factors are likely to have contributed.","Social Security Earnings Test","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2008,http://jhr.uwpress.org/content/43/1/57.refs,"Employment and Training Review Protocol" "Teaching self-sufficiency: An impact and benefit-cost analysis of a home visitation and life skills education program. Findings from the rural welfare-to-work strategies demonstration evaluation","Meckstroth, A., Burwick, A., & Moore, Q. (2008). Teaching self-sufficiency: An impact and benefit-cost analysis of a home visitation and life skills education program. Findings from the rural welfare-to-work strategies demonstration evaluation. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Building Nebraska Families (BNF) rural welfare-to-work program on employment, earnings, and public benefit receipt. The authors randomly assigned Nebraska Temporary Assistance for Needy Families (TANF) recipients identified as hard-to-employ to either a treatment group, which received in-home life skills and family management training, or a control group, which did not. The authors evaluated the treatment’s impact using Nebraska state administrative data and follow-up surveys. The study found that BNF increased the average likelihood of ever being employed in the first year after enrollment by 10.8 percentage points and that of ever being employed in the first 30 months after enrollment by 6.3 percentage points. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the BNF rural welfare-to-work program and not to other factors.","Building Nebraska Families (BNF)","Other employment and reemployment","Low income",,"United States",2008,http://www.acf.hhs.gov/sites/default/files/opre/teaching_self.pdf,"Employment and Training Review Protocol" "Choice architecture and retirement savings plans","Benartzi, S., Peleg, E., & Thaler, R. (2007). Choice architecture and retirement savings plans. Los Angeles, Ca. SSRN working paper.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The report’s objective was to examine the impact of choice architecture—that is, aspects of a savings plan’s structure and enrollment process—on 401(k) savings decisions. The authors discussed five original, observational studies. Two studies conducted regression analyses using survey data. One used an interrupted time series design and data from Vanguard on employees at 13 companies. Two additional studies conducted regression analyses using Vanguard data on 1.5 million participants in 1,830 of Vangaurd’s defined contribution retirement plans. The study receiving a moderate rating (study 5) found that members of retirement plans that offer retirement date funds, relative to members of retirement plans that offer risk-based funds or neither, held comparatively more equity when they were younger and comparatively less equity when they were older. The quality of causal evidence presented in this report is moderate because study 5 was a quasi-experimental design with adequate controls. We must note, however, that the remaining four studies provide causal evidence rated as low. The overall moderate evidence rating means we are somewhat confident that the estimated effects are attributable to the employees’ access to asset allocation funds, but other factors might also have contributed. In the case of the remaining four studies, however, we are not confident that the estimated effects are attributable to the interventions analyzed. Other factors are likely to have contributed.","the Choice Architecture of Retirement Savings Plans","Employer provided retirement benefits",Employed,,"United States",2007,http://164.67.163.139/Documents/areas/fac/accounting/Benartzi_ChoiceArchitectur…,"Behavioral Finance: Retirement Review Protocol" "Individual account investment options and portfolio choice: Behavioral lessons from 401(K) plans","Brown, J., Lang, N., & Weisbenner, S. (2007). Individual account investment options and portfolio choice: Behavioral lessons from 401(K) plans. National Bureau of Economic Research working paper No. 13169. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study aimed to determine whether the number of fund options offered within a given asset class (for example, domestic equities) by a 401(k) plan influences the types of investments made by plan members. The authors analyzed differences in portfolio allocations based on the number of funds available in these asset classes using regression analysis, controlling for other 401(k) and firm characteristics. The analysis demonstrated that as the number of investment options within an asset class increase, individuals tend to allocate more of their 401(k) savings to funds within that class. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that an increase in the number of funds offered within a class causes the increase in the share of funds allocated to assets in that class. We can only say that the two variables of interest are positively correlated.","Varying the Number of Risky Choices","Employer provided retirement benefits",Employed,,"United States",2007,http://www.nber.org/papers/w13169,"Behavioral Finance: Retirement Review Protocol" "The Quantum Opportunity Program demonstration: Final impacts","Schirm, A., Stuart, E., & McKie, A. (2006). The Quantum Opportunity Program demonstration: Final impacts. Washington, DC: Mathematica Policy Research.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-No impacts Education and skills gains Employer benefits receipt-Mod/high-No impacts Employer benefits receipt","Summary: The report’s objective was to evaluate the impact of the Quantum Opportunity Program (QOP)—which included case management and mentoring, education, developmental activities, community service, supportive services, and financial incentives—on educational attainment and labor market participation nine years after program enrollment. Earlier reports presented impact findings over a shorter follow-up period.  About 1,100 9th-grade students from 11 high schools were randomly assigned to the treatment group, which could enroll in QOP, or the control group, which could not participate in QOP activities. The study team administered a survey to treatment and control group members about nine years after program enrollment (six years after their scheduled graduation from high school) to collect information on education and labor market outcomes. The study reported no statistically significant impacts of participation in QOP on postsecondary educational attainment, likelihood of employment, or earnings for the full sample. The quality of the causal evidence presented in this report is high because it was a well-implemented randomized controlled trial. This means we are confident that estimated impacts would be solely attributable to QOP, and not other factors.","Quantum Opportunities Program (QOP)","Mentoring Other training and education Youth programs",Youth,,"United States",2006,http://www.mathematica-mpr.com/~/media/publications/PDFs/QOPfinalimpacts.pdf,"Opportunities for Youth Review Protocol" "Are empowerment and education enough? Underdiversification in 401(k) plans","Choi, J., Laibson, D., & Madrian, B. (2005). Are empowerment and education enough? Underdiversification in 401(k) plans. Brookings Papers on Economic Activity 2005, (2), 151-213.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-No impacts Employer benefits receipt","Summary: The study examined factors that can affect diversification from employer stock in 401(k) plans. The study analyzed the impact of loosened diversification restrictions and media coverage of highly publicized events on 401(k) holdings in employer stock. The authors assessed factors affecting diversification in 401(k) plans through four studies. The first was an interrupted time series analysis of employer stock holdings at an unnamed company. The other three studies explored the impact of media attention to the Enron, WorldCom, and Global Crossing bankruptcies through regression analysis. All analyses were based on transaction-level and annual data from 1997 to 2003 from Hewitt Associates, a benefits administration company. The authors attributed only a 2.4 percentage point decline in employer stock holdings to news coverage of the Enron, WorldCom, and Global Crossing bankruptcies across the entire Hewitt 401(k) Index. The quality of causal evidence presented in this report is low. In the first study, the authors did not account for prior trends in the outcome variable and in the second through fourth studies they did not provide information a reader would need to assess changes in sample composition. This low rating means we are not confident that the estimated effects are attributable to any of the interventions studied, whether loosened diversification restrictions or media coverage of the Enron, Global Crossing, and WorldCom bankruptcies. Other factors are likely to have contributed.","Factors Promoting 401(k) Diversification","Employer provided retirement benefits",Employed,,"United States",2005,,"Behavioral Finance: Retirement Review Protocol" "Save More Tomorrow™: Using behavioral economics to increase employee saving","Thaler, R., & Benartzi, S. (2004). Save More Tomorrow™: Using behavioral economics to increase employee saving. Journal of Political Economy, 112(S1), S164-S187.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study aimed to determine whether a retirement savings plan that automatically increased 401(k) contributions whenever an enrolled employee’s salary increased improved savings rates among employees. The authors compared savings rates among employees who opted to participate in the plan with savings rates among employees who declined to join the plan. Across the three experimental sites, average savings rates tended to increase, with higher increases for program participants than for nonparticipants. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that the difference in savings rates between program participants and nonparticipants results from participation in the savings plan.","Save More Tomorrow™","Employer provided retirement benefits",Employed,,"United States",2004,,"Behavioral Finance: Retirement Review Protocol" "Saving for retirement on the path of least resistance","Choi, J., Laibson, D., Madrian, B., and Metrick, A. (2004). Saving for retirement on the path of least resistance. Working paper. Cambridge, MA.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study examined the impact of different 401(k) plan designs on employees’ saving behavior at several U.S.-based firms. The analysis was based on administrative data provided by each company. The study used five analyses—one based on an interrupted time series (ITS) design, one based on a randomized controlled trial (RCT), and three quasi-experimental regression analyses—to estimate impacts on 401(k) participation and contribution rates. The study found that employees’ decision making was influenced by defaults, such as automatic enrollment and automatic disbursements from 401(k) plans. It also found that the gap in 401(k) participation rates between new employees facing a mandatory waiting period for eligibility and those with immediate eligibility disappeared shortly after the former group gained eligibility. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the various 401(k) program designs evaluated in this study. Other factors are likely to have contributed.","401(k) Plan Design Changes","Employer provided retirement benefits",Employed,,"United States",2004,http://www.hks.harvard.edu/fs/bmadria/Documents/Madrian%20Papers/Saving%20for%2…,"Behavioral Finance: Retirement Review Protocol" "For better or for worse default effects and 401(k) savings behavior","Choi, J.J., Laibson, D., Madrian, B.C., & Metrick, A. (2004). For better or for worse default effects and 401(k) savings behavior. National Bureau of Economic Research, 81-126","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s key objective was to examine the impact of automatic 401(k) plan enrollment on enrollment rates and contribution levels at three firms in the United States. The analysis was based on data on enrollment in and contributions to retirement plans from the companies themselves. The study used an interrupted time series (ITS) design, analyzing changes in outcomes before and after the implementation of automatic enrollment. The study found that automatic enrollment increased participation rates. In addition, automatic enrollment led to a larger percentage of participants investing at the plan’s default contribution rate. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the companies’ automatic enrollment policies. Other factors are likely to have contributed.","Automatic Enrollment","Employer provided retirement benefits",Employed,,"United States",2004,,"Behavioral Finance: Retirement Review Protocol" "How much choice is too much? Contributions to 401 (k) retirement plans","Iyengar, S.S., Huberman, G., & Jiang, W. (2003). How much choice is too much? Contributions to 401 (k) retirement plans. Pension Research Council working paper.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Unfavorable impacts Employer benefits receipt","Summary: This study’s objective was to examine whether the number of fund options offered by a 401(k) retirement plan influenced employees’ decisions to enroll in the plan. The authors analyzed employees’ participation rates in 401(k) plans using administrative data provided by an investment management company and regression analysis, controlling for both employee- and plan-level characteristics. The analysis demonstrated that as the number of fund options increased, employees became less likely to enroll in a 401(k) plan. Adding one fund to the list of investment options was associated with a decrease in participation of 0.15 to 0.20 percentage points. The quality of causal evidence presented in this study is low. This means that we are not confident that the increase in the number of funds offered caused the decrease in 401(k) enrollment. Other factors are likely to have contributed to the observed relationship.","the Number of Fund Choices","Employer provided retirement benefits",Employed,,"United States",2003,,"Behavioral Finance: Retirement Review Protocol" "The role of information and social interactions in retirement plan decisions: Evidence from a randomized experiment","Duflo, E., & Saez, E. (2003). The role of information and social interactions in retirement plan decisions: Evidence from a randomized experiment. Quarterly Journal of Economics, 118(3), 815–842.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine the impact of receiving information about tax-deferred retirement accounts (TDAs) on enrollment in the accounts. The study used a randomized controlled trial (RCT) at a large university. Departments within the university were randomly assigned into two groups. In the treatment group, nonfaculty employees were randomly selected to receive or not receive a letter offering a $20 incentive to attend an employee-benefits fair, at which information on TDAs was provided. In the other group of departments, no employees received a letter. The study examines both the effect of receiving the letter or being in a department in which some people received the letter. Under most model specifications, the study found that people in both treatment groups had higher enrollment in TDAs, as measured 4.5 and 11.0 months after the fair. The quality of the causal evidence presented in this study is high. This means we are confident that the estimated effects of the intervention are attributable to the intervention itself, and not some other factor.","Incentives for Attending a Benefits Fair","Employer provided retirement benefits",Employed,,"United States",2003,,"Behavioral Finance: Retirement Review Protocol" "The power of suggestion: Inertia in 401(k) participation and savings behavior","Madrian, B. C., & Shea, D. F. (2001). The power of suggestion: Inertia in 401(k) participation and savings behavior. The Quarterly Journal of Economics, 116(4), 1149-1187.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Mixed impacts Employer benefits receipt","Summary: The study examined the impact of two simultaneously implemented changes in the 401(k) plan of a large, Fortune 500 company: (1) automatic enrollment into the plan immediately after hire (a change from requiring employees to opt in to the plan to requiring them to opt out); and (2) removing the one-year tenure requirement for employees to participate in the 401(k) plan, enabling immediate contributions for employees with less than one year of tenure at the firm. The study used employee-level data from the company. It used an interrupted time series design to compare the participation and contribution rates of those hired before the company’s 401(k) plan changes to the behaviors of those hired after the changes were implemented. The authors compared employees subject to changes (1) and (2) to those subject to change (2) only and those subject to change (2) only to those subject to neither change. Employees subject to both immediate contributions and automatic enrollment had higher 401(k) participation rates and lower 401(k) contribution rates than those subject to immediate contributions only. Employees subject to immediate contributions had only slightly higher participation rates compared with those not subject to either plan change. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the company’s changes to its 401(k) plan design. Other factors are likely to have contributed.","Allowing Immediate 401(k) Contributions and Automatic Enrollment","Employer provided retirement benefits",Employed,"Large business","United States",2001,,"Behavioral Finance: Retirement Review Protocol" "Investor behavior and the purchase of company stock in 401(k) plans—the importance of plan design","Liang, N., & Weisbenner, S. (2002). Investor behavior and the purchase of company stock in 401(k) plans—the importance of plan design. National Bureau of Economic Research working paper 9131. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine whether 401(k) plan characteristics, including the number of total investment options offered by a 401(k) plan and the requirements for whether employers’ matching contributions had to be allocated to employer stock, influenced the percentage of employees’ 401(k) plan contributions allocated to employer stock. The authors analyzed differences in portfolio allocations based on the number of funds available and other key plan characteristics using data from Compustat and Securities and Exchange Commission (SEC) forms and regression analyses, controlling for firm-fixed effects. The number of investment options and the percentage of assets allocated to employer stock were negatively correlated. Further, employees at firms that required employer matching contributions to be allocated to employer stock allocated a larger share of assets to employer stock than those at other firms. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that differences among plan characteristics caused changes in the share of funds allocated to employer stock. Other factors are likely to have contributed.","Number of Fund Choices and Restrictions on Employer Matching Contributions","Employer provided retirement benefits",Employed,,"United States",,,"Behavioral Finance: Retirement Review Protocol"