This study was conducted by staff from Abt Associates, which co-administers CLEAR. The review of this study was conducted by ICF, which co-administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.
Citation
Yang, H., Freiman, L., & Lubell, J. (2021). Early Assessment of the Impacts of Multifamily Family Self-Sufficiency (FSS) Programs administered by Compass Working Capital in Partnership with Preservation of Affordable Housing (POAH). Cambridge, MA: Abt Associates.
Highlights
- The study's objective was to examine the impact of the Compass Family Self-Sufficiency (FSS) Program on earnings and public benefits receipt outcomes.
- The study used a nonexperimental design to compare the outcomes of households enrolled in the Compass FSS Program to a matched comparison group of households in the same regions. The authors used administrative data and statistical models to compare the outcomes of the treatment and comparison households.
- The study found a significant relationship between Compass FSS Program participation and lower amounts of annual public assistance.
- This study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to the Compass Family Self-Sufficiency (FSS) Program; other factors are likely to have contributed.
Intervention Examined
Compass FSS Program
Features of the Intervention
The Family Self-Sufficiency (FSS) program was created by Congress in 1990 and is managed by the U.S. Department of Housing and Urban Development (HUD). Its purpose is to help households that receive housing assistance through two HUD programs (Housing Choice Voucher program and Public Housing) achieve economic security. The program’s three primary goals include ensuring stable and affordable housing, providing case management and employment coaching, and creating an escrow account that grows with participants’ earnings and rent contributions.
In 2015, Congress expanded the FSS program to include Project-Based 8 rental assistance programs. Compass Working Capital implemented a model that aligns with the original goals of the FSS program while adding new features. These include marketing and outreach to help families set and achieve goals, a participant-driven coaching approach, a public-private funding model, and a focus on enhancing financial capacity, savings, debt reduction, budgeting, and credit scores. After the 2015 authorization, the Compass FSS program became available to families living in Preservation of Affordable Housing’s (POAH) properties in New England and Missouri.
Features of the Study
The study used a nonexperimental design to examine the impact of the Compass FSS Program on earnings and public benefit receipt outcomes. Compass FSS Program households living in six Project-Based Section 8 multifamily POAH communities in New England (Massachusetts, Rhode Island, and Connecticut) were compared to a matched group of similar households living in Section 8 Affordable Multifamily Housing within the same regions and time frame. The treatment group consisted of 81 households who received the Compass FSS Program. Households in comparison conditions did not receive the Compass FSS Program services but were eligible to participate in any specific programs offered through their non-POAH provided Project-Based Section 8 affordable housing communities.
The primary data source was HUD's Tenant Rental Assistance Certification System (TRACS) data system. This data system provided administrative data about households living in multifamily Project-Based Section 8 Housing properties and collects updated information annually when households recertify their income to remain in the program. Heads of households included in the analysis had to be under 62 years old and not have a disability at the time of enrollment. The authors used statistical models to compare the outcomes of treatment and comparison group households.
Findings
Earnings and wages
- The study found no statistically significant differences in annual earnings between treatment and comparison group households in POAH New England properties.
Public benefits receipt
- The study found that treatment group households in the POAH New England properties received significantly less annual public assistance income, including Temporary Assistance for Needy Families (TANF) and other government assistance, than comparison group households.
- However, the study did not find significant differences between the groups in the amount of Supplemental Security Income (SSI), Social Security, and pension received or in the amount of other annual income received (e.g., child support, unemployment insurance benefits).
Considerations for Interpreting the Findings
The authors matched the treatment and comparison households on several variables including racial and ethnic composition, years of program participation, age of household head at baseline, earnings amount at baseline, public assistance amount at baseline, other income amount at baseline, and Social Security amount at baseline. However, the author did not match on sex or control for it in the statistical analyses as required by the protocol. These preexisting differences between the groups—and not the Compass FSS Program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs. Also, the study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure the groups were similar before participation in the program. This means we are not confident that the estimated effects are attributable to the Compass Family Self-Sufficiency (FSS) Program; other factors are likely to have contributed.