Absence of conflict of interest.
Citation
Chapman, A. (2017). Quantitative analysis of an urban community college S-STEM program. (Unpublished doctoral dissertation). University of Phoenix, Phoenix, AZ.
Highlights
- The study’s objective was to examine the impact of the Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) program on community college students’ success and progress.
- The study used a nonexperimental design to compare the outcomes of S-STEM program participants to a matched comparison group. Using administrative data from the community college, the author tested for group differences in student success and progress rate.
- The study found that participation in the S-STEM program was associated with significantly higher rates of student success and progress.
- The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the S-STEM program, but other factors might also have contributed.
Intervention Examined
The Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) program
Features of the Intervention
The Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) program was funded by the National Science Foundation (NSF) and designed to support academically talented low-income students. The program had three primary goals: to increase participants’ retention, success, and completion. Participants in the program were awarded a $3,500 scholarship per semester for up to four semesters. The program also offered authentic experiences (e.g. internships and undergraduate research), a faculty mentor, laboratory or industry mentoring, free tutoring services, customized academic advisement, virtual cohort participation, presentations of research, and a student support coordinator. Eligibility requirements included academic excellence, financial need, and full-time enrollment in a science, technology, engineering, or mathematics degree program (associate, baccalaureate, or graduate level). Participation required an application and acceptance into the S-STEM program.
Features of the Study
The author evaluated the effects the S-STEM program on student success and progress rate at one urban community college using a nonexperimental study. Study participants included students majoring in STEM fields who attended the community college from the Fall of 2012 through the Fall of 2015. The author matched treatment and comparison students on age, financial need, gender, and ethnicity. After matching, students were assigned a random number identifier (RNI). The students with the lowest RNI were selected to the comparison group. The analysis sample included a total of 135 students; 45 students receiving the S-STEM intervention program and 90 comparison STEM students. The author obtained demographic and academic data from the community college. Outcomes included student success (defined as degree completion, retention, or university transfer) and progress rates (defined as credits earned divided by semesters attended). The author used chi-square and t-test statistics to examine differences between the groups.
Findings
Education and skills gain
- The study found a significant relationship between program participation and student success, where the odds of success for S-STEM students were almost 5 times higher than for the comparison group.
- The study found that students in the S-STEM program had significantly higher progress rates (13.3) than students in the comparison group (11.5).
- There was also a significant difference in the progress rates of the S-STEM group before and after the program—an average of almost three credit hours per semester—indicating that the students' progress rates increased after participation in the program.
Considerations for Interpreting the Findings
Due to the older age of the study sample and their non-traditional status (e.g., immigrant, GED completion), the author expressed that high school grade point average or rank was not appropriate as a pre-intervention measure of academic achievement. Therefore, random selection of the comparison group was used to control for high school GPA and class rank. Although the author used a well-implemented nonexperimental design, she did not account for differences between groups’ academic achievement histories before program participation. These preexisting differences between the groups—and not the S-STEM program—could explain the observed differences in outcomes.
Additionally, the data used in the progress rate analyses had non-normal distributions and clear outliers. Several transformations and modifications were made in order to normalize the data. This could have had an impact on the significance of the findings.
Causal Evidence Rating
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the S-STEM program, but other factors might also have contributed.