Citation
Drane, D., Micari, M., & Light, G. (2014). Students as teachers: Effectiveness of a peer-led STEM learning programme over 10 years. Educational Research & Evaluation, 20(3), 210-230.
Highlights
- The study’s objective was to examine the impact of the Gateway Science Workshop (GSW) program at a university in the United States on course grades.
- Using administrative data on course grades from more than 20,000 students in biology, chemistry, physics, engineering, and math courses, the authors estimated regression models to determine the impact of the GSW program.
- The study found no significant relationship between GSW program participation and students’ grades for either gender.
- The quality of causal evidence presented in this report is low because the authors did not adequately adjust for existing differences between treatment and comparison students. This means we are not confident that the estimated effects are attributable to the GSW program. Other factors are likely to have contributed.
Intervention Examined
The Gateway Science Workshop Program
Features of the Intervention
All freshmen and sophomore students enrolled in biology, chemistry, organic chemistry, physics, math, and engineering undergraduate courses at a selective university in the United States from fall 2001 to spring 2011 were invited to participate in the GSW program. Students chose whether to participate. The GSW program was a year-long peer-led group in which students met for two hours a week to work on challenging problems related to their coursework. Each group consisted of five to seven students, facilitated by a trained undergraduate peer.
Features of the Study
The authors used a non-experimental analysis. The authors considered all students who elected to participate in the program and attended all but two of the workshops during the first quarter to be in the treatment group. All students who did not join the GSW group or who missed more than two workshops were considered the comparison group. Overall, there were 4,055 treatment and 15,989 comparison students. School administrative data provided students’ course grades. The authors estimated regression models comparing the outcomes of treatment and comparison group members, controlling for gender and race.
Findings
- The study found no significant relationships between GSW program participation and students’ grades when modeling outcomes separately by gender.
Considerations for Interpreting the Findings
The authors did not establish the comparability of students in the treatment and comparison groups, so the estimated impact of the GSW program might reflect the effect of other factors in addition to the effect of the intervention. Although the authors stated that they estimated their regression models controlling for prior academic achievement, they did not present these findings in the report. Instead, they reported results from regression models that controlled only for gender and race. In addition, study participants chose whether to participate in the intervention, so there could have been unobservable differences (for example, in levels of motivation) between students who chose to participate in the GSW program and those who did not. Therefore, the estimated impacts might reflect differences in unobservable characteristics rather than the true impact of the program itself.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not adequately adjust for existing differences between treatment and comparison students. This means we are not confident that the estimated effects are attributable to the GSW program. Other factors are likely to have contributed.