Citation
Sommo, C., Boynton, M., Collado, H., Diamond, J., Gardenhire, A., Ratledge, A., . . . Weiss, M. J. (2014). Mapping success: Performance-based scholarships, student services, and developmental math at Hillsborough Community College. New York: MDRC.
Highlights
- The study’s objective was to examine the impact of the Mathematics Access Performance Scholarship (MAPS) on completion of developmental and college-level math, credits attempted and earned, and retention at two Hillsborough Community College campuses in Florida (the Dale Mabry campus and the Brandon campus).
- Developmental math students were randomly assigned to the MAPS program or to the control group. College transcripts were used to assess outcomes.
- The study found that students in the treatment group attempted and earned more math credits and were more likely to complete Intermediate Algebra (a developmental math course).
- The quality of causal evidence provided in this study is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the MAPS program and not to other factors.
Intervention Examined
The Mathematics Access Performance Scholarship (MAPS)
Features of the Intervention
The treatment condition involved participating in the MAPS program at two Hillsborough Community College campuses in Florida (the Dale Mabry campus and the Brandon campus). The program offered a performance-based scholarship to students who completed a sequence of three math courses across consecutive semesters. The total scholarship available was $1,800, which was incrementally awarded when students remained enrolled in each course past the add/drop period ($100) and when they successfully completed each course, including any course requirements for visiting the Math Lab for assistance ($500). Students in the treatment condition could also earn a textbook or book voucher bonus (worth about $150-200) if they received a grade of a B or better in each math course. Students also received periodic email reminders about program requirements.
Features of the Study
Students were randomly assigned after agreeing to participate and completing a baseline survey. The study used an assignment ratio of 3:2 treatment to control and kept this ratio consistent across the three cohorts. Impacts were estimated by comparing treatment and control group means at multiple time points, using t tests for hypothesis testing.
Findings
- Students in the treatment group attempted 1 more math credit and earned 0.7 more math credits than students in the control group.
- Students in the treatment group were 5.4 percentage points more likely than control students to complete Intermediate Algebra as their highest-level math course by the end of two years.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to progress toward degree completion. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence provided in this study is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the MAPS program and not to other factors.