Citation
Kniola, D., Chang, M., & Olsen, D. (2012). Transformative graduate education programs: An analysis of impact on STEM and non-STEM Ph.D. completion. Higher Education, 63(4), 473–495. doi:10.1007/s10734-011-9453-8
Highlights
- The study’s objective was to determine whether universities participating in Transformative Graduate Education Programs (TGPs) conferred a greater number of science, technology, engineering, and mathematics (STEM) doctoral degrees, particularly to women and underrepresented candidates of color. The study also examined whether TGPs resulted in a greater number of non-STEM doctoral degrees, but that is not the focus of this review.
- The authors analyzed data from the Integrated Postsecondary Education Data System to estimate the impact of a university’s participation in one of seven TGPs during the 2003–2005 period, controlling for institutional characteristics.
- The study found that TGPs were associated with a higher number of doctoral degrees awarded in STEM fields to women across all institutions.
- The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to universities’ participation in TGPs. Other factors are likely to have contributed.
Intervention Examined
Transformative Graduate Education Programs (TGPs)
Features of the Intervention
Despite growing graduate enrollments of women and underrepresented candidates of color, STEM doctoral conferral rates for these students remain sluggish. In response, prestigious foundations and institutional associations launched several grants to fund TGPs. Competitive TGP grants challenge universities to make doctoral programs more inclusive and supportive, offering more well-rounded preparation to meet the real-world demands of contemporary careers in research, teaching, and service.
Features of the Study
The authors examined the effectiveness of 7 well-known TGPs sponsored by prominent institutions among 246 doctoral degree–granting universities in the United States from 2003–2005. Of those, 117 did not participate in a TGP and 129 participated in at least one TGP. The TGPs studied included (from largest to smallest):
- National Science Foundation’s Integrative Graduate Education and Research Traineeship (IGERT) program (87 institutions)
- Council of Graduate Schools’ PhD Completion Project (43 institutions)
- Council of Graduate Schools and Association of American Colleges & Universities’ Preparing Future Faculty (43 institutions)
- Carnegie Foundation for the Advancement of Teaching’s Initiative on the Doctorate (42 institutions)
- National Science Foundation’s ADVANCE (24 institutions)
- Woodrow Wilson National Fellowship Foundation’s Responsive PhD (20 institutions)
- National Institutes of Health’s Graduate Partnerships program (10 institutions)
The authors employed hierarchical ordinary least squares regression models to predict the 2003–2005 average annual number of STEM doctoral degrees, STEM doctoral degrees awarded to women, and STEM doctoral degrees awarded to underrepresented students of color. They controlled for institution-level differences in institutional control, average annual student enrollment, level of research activity, and proportion of doctoral degrees conferred in STEM disciplines. The authors also compared the relative effectiveness of the four largest TGPs.
Findings
- The study found that TGPs were associated with a higher number of doctoral degrees awarded in STEM fields to women across all institutions; however, this association did not hold when looking only at STEM-dominant institutions.
- When looking at the four largest TGPs, only IGERT was associated with a higher number of doctoral degrees awarded to women in STEM fields.
Considerations for Interpreting the Findings
Although the study controlled for several important institutional-level variables, the authors acknowledged the possibility that other institutional characteristics that were not controlled for in the model could encourage both the adoption of TGPs and the award of more doctoral degrees to women. For instance, institutional leadership and culture could drive both the desire to adopt a TGP and higher rates of doctoral degree completion by women. In addition, even though CLEAR does not require explicit demonstrations of baseline equivalence, the study indicated that the groups being compared were quite dissimilar, leading to doubts about the extent to which including controls in the model could reduce the potential for bias.
Causal Evidence Rating
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to universities’ participation in TGPs. Other factors are likely to have contributed.