Citation
Campbell, B., Robb, S., Abbott, S., & Mutunga, S. (2014). Impact of a 5-week collegiate-level residential STEM summer program on secondary school students (Research to practice). Proceedings of the 2014 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.
Highlights
-
- The study’s objective was to examine the impact of participating in the Pennsylvania Governor’s School for the Sciences’ (PGSS) five-week residential program in summer 2013 on high school seniors’ attitudes toward and interest in science, technology, engineering, and math (STEM).
- Using surveys developed for the study, the authors compared the outcomes of students before and after participating in the PGSS program.
- The study found that female students had a significantly improved disposition toward technology and general science and a significantly higher interest in the specific fields of chemistry, computer science, and math at the end of the program, compared to the start of the program.
- The quality of causal evidence presented in this report is low because the authors did not observe outcomes for multiple periods before the intervention nor account for selection into the intervention. This means we are not confident that the estimated effects are attributable to the PGSS program; other factors are likely to have contributed.
Intervention Examined
The Pennsylvania Governor’s School for the Sciences (PGSS)
Features of the Intervention
The PGSS program is a five-week program held at Carnegie Mellon University for students in the summer before their senior year of high school. Five days a week, students take core classes in biology, chemistry, computer science, math, and physics; complete lab requirements; and work collaboratively with other students on projects and homework assignments. Courses cover college-level content, and the program emphasizes collaboration among classmates. No classes are held on weekends; students typically use their time for social activities and homework. Students successfully completing the program receive a personalized letter of recommendation from faculty and teaching assistants. The study examined the program during summer 2013, during which 30 female and 30 male students participated.
Features of the Study
The authors compared the outcomes of students before and after participating in the PGSS program. Using a survey developed for the study, the authors collected information on students’ attitudes toward and interest in STEM on the first day of the program and again during the last week. Results were presented separately for boys and girls.
Findings
-
- The study found that female students gave technology and general science a more favorable rating at the end of the program than at the beginning of the program.
- Female students had a statistically significantly higher interest in the specific fields of chemistry, computer science, and math at the end of the program, compared to their stated interest at the start of the program.
Considerations for Interpreting the Findings
The authors compared the outcomes of participants measured once before and once after they participated in the PGSS program. CLEAR’s guidelines require that the authors observe outcomes for multiple periods before the intervention to rule out the possibility that participants had increasing or decreasing trends in the outcomes examined before enrollment in the program. That is, if participants who had increasing interest in and disposition toward STEM fields tended to enroll in the program, we would anticipate further improvements in their attitudes toward STEM over time, even if they did not participate in the program. Without knowing the trends before program enrollment, we cannot rule this out.
Further, CLEAR’s guidelines require that the authors demonstrate that participants would not have selected into the intervention based on pre-intervention trends in the outcomes and/or their own characteristics. Because PGSS participants comprise only those students who applied for and were admitted to the program, they are likely different from eligible students who were not admitted or did not apply. It is possible that the characteristics of the participants led to the results, not the program itself.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not observe outcomes for multiple periods before the intervention nor account for selection into the intervention. This means we are not confident that the estimated effects are attributable to the PGSS program; other factors are likely to have contributed.