Absence of conflict of interest.
Citation
Mayer, A. K., Patel, R., & Gutierrez, M. (2016). Four-year degree and employment findings from a randomized controlled trial of a one-year performance-based scholarship program in Ohio. Journal of Research on Educational Effectiveness, 9(3), 283-306. doi: 10.1080/19345747.2015.1086914
Highlights
- The study’s objective was to examine the impact of a performance-based scholarship program on community college students’ credit accumulation, degree/certificate completion, employment, and earnings.
- The study was a randomized controlled trial conducted at three community colleges in Ohio. Eligible students were randomly assigned to either the treatment or control groups. The authors conducted statistical models to examine differences in outcomes between the groups four years post-random assignment.
- The study found that when compared to the control group, students in the performance-based scholarship program had significantly higher credit accumulation and rates of degree/ certificate completion.
- The quality of causal evidence presented in this study is high because it is based on a well-implemented randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the one-year performance-based scholarship program and not to other factors.
Intervention Examined
The performance-based scholarship program
Features of the Intervention
A one-year performance-based scholarship program was implemented at three community colleges in Ohio during the 2008-2009 academic year: Lorain County Community College, Owens Community College, and Sinclair Community College. First-time and continuing students who were eligible for financial aid were also eligible to receive the scholarship. The program offered performance-based scholarships of up to $900 per semester (or $600 per quarter) if students maintained a C average or better and were enrolled in at least 12 credit hours. For students enrolled part-time, the program offered scholarships of $450 per semester (or $300 per quarter) if students maintained a C average or better and were enrolled in 6 to 11 credit hours. Scholarships were paid directly to the students and were designed to supplement other sources of financial aid.
Features of the Study
The study used a randomized controlled trial to examine the impact of performance-based scholarships on credit accumulation, degree/certificate completion, employment, and earnings. The sample was drawn from the colleges’ financial aid databases. Eligible first-time or continuing students were invited to attend an information session about the scholarship program and the study. Of the 2,285 students who attended the information session and completed the required paperwork, 1,359 students were randomly assigned to the treatment group and 926 were assigned to the control group. Most participants were low-income mothers, and the majority received government benefits such as TANF and food stamps. Data were obtained through multiple sources including a baseline information form completed at the information session, Free Application for Federal Student Aid (FAFSA) records, community college financial aid records, student transcripts, the National Student Clearinghouse data, and data from the Ohio Department of Job and Family Services. The authors used statistical models to estimate the effects of the program, controlling for cohort and campus.
Findings
Education and skills gain
- The study found a significant impact of the performance-based scholarship program on students’ credit accumulation. During the first semester, 33.2% of treatment students earned 12 or more credits compared to 26.3% of control group students. During the second semester, 28.8% of treatment students earned 12 or more credits compared to 18.3% of control group students.
- The study found a significant impact of the performance-based scholarship program on students’ degree/certificate completion in years two and three of the study. In year 2, 20.5% of treatment students completed a degree/certificate compared to 17.2% of control group students. In year 3, 28.3% of treatment group students and 24.5% of control group students completed a degree/certificate, respectively.
Earnings and wages
- The study did not find a significant impact of the performance-based scholarship program on students’ earnings.
Employment
- The study did not find a significant impact of the performance-based scholarship program on students’ employment outcomes.
Considerations for Interpreting the Findings
The program was offered to both first-time students and continuing students. However, the authors were unable to distinguish first-time students from continuing students and could not determine whether the effects were different for these two groups. Also, the duration of the program differed across sites with the program being offered for two semesters at Lorain and Owens and for three semesters at Sinclair.
Causal Evidence Rating
The quality of causal evidence presented in this study is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the one-year performance-based scholarship program and not to other factors.