Absence of conflict of interest.
Citation
Huang, M., & Yamada, H. (2017). Maintaining success rates: Does Statway® sustain its impact as it scales to new classrooms and institutions? Carnegie Foundation for the Advancement of Teaching: Stanford, CA.
Highlights
- The study’s objective was to examine the impact of the Statway® math program on community college students’ earning of math credits.
- The authors used a nonexperimental design to compare the outcomes of students who were in the Statway® program to a matched comparison group.
- When compared to non-participating students, the study found that Statway® program participation was associated with a significantly higher likelihood of earning college math credit.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Statway® program; other factors are likely to have contributed.
Intervention Examined
The Statway® math program
Features of the Intervention
The Statway® program was developed by the Carnegie Foundation for the Advancement of Teaching with a goal of improving success rates for students in developmental math. Statway® provides an accelerated program to allow students to meet math requirements by providing a two-course program, intended for students to take consecutively. The program includes elementary and intermediate algebra as well as college-level statistics for students pursing non-STEM majors. The curriculum is grounded in evidence-based practices to promote learning such as productive struggle, explicit connections, and deliberate practice. Socioemotional supports are integrated into the program with productive persistence to help support and build student confidence in math. Language and literacy supports provide students with information on math vocabulary and core concepts and skills to help with critical thinking and reasoning. Finally, the program includes faculty professional development and a "networked improvement community" (NIC) to allow stakeholders to collaborate across institutions.
Features of the Study
The authors used a nonexperimental design to compare the outcomes of students who were in the Statway® program with students who had taken traditional developmental math. The study included students from two Statway® cohorts. The Year 3 Cohort (Fall 2013) included students from 19 community colleges and three 4-year universities across seven states (California, Connecticut, Florida, Minnesota, New York, Texas, and Washington). The Year 4 Cohort (Fall 2014) included students from 22 community colleges and four 4-year universities across 10 states (including the addition of Hawaii, Massachusetts, and Wyoming). The authors received student demographic, prior course enrollment, and performance data from each institution. The authors used a hierarchical linear model framework when creating propensity scores to match students on 32 characteristics. Students in the Year 3 cohort were matched to students from Fall 2012 and those in the Year 4 Cohort were matched to students from Fall 2013. The Year 3 sample included 1,353 students in the treatment group and 5,853 in the comparison group. The Year 4 sample included 1,254 students in the treatment group and 4,240 in the comparison group.
Study Sites
- California
- Connecticut
- Florida
- Hawaii
- Massachusetts
- Minnesota
- New York
- Texas
- Washington
- Wyoming
Findings
Education and skills gain
- The study found a significant relationship between program participation and earning of math credits, where the odds of earning college math credit for Statway® students were almost five times higher than for the comparison group.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to Statway® students. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not the Statway® program—could explain the observed differences in outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Statway®; other factors are likely to have contributed.