Citation
Zajicek, A., Morimoto, S., Terdalkar, A., Hunt, V., Rencis, J., & Lisnic, R. (2011). Recruitment strategies for gender equity: Lessons from cohort 1 and cohort 2 advance institutions. Washington, DC: American Society for Engineering Education.
Highlights
- The study’s objective was to evaluate the effectiveness of the ADVANCE institutional transformation program (ADVANCE-IT), sponsored by the National Science Foundation (NSF), on the recruitment of female engineering faculty at participating U.S. colleges and universities.
- The authors used program implementation and outcome data obtained from seven participating colleges and universities to compare the number of female engineering faculty before and after institutions participated in the intervention.
- The study found no statistically significant relationships between participation in the ADVANCE-IT program and the number of female assistant professors at those institutions.
- The quality of causal evidence presented in this study is low because the authors conducted no tests of statistical significance and did not observe outcomes for multiple periods before the intervention. This means we are not confident that the estimated effects would have been attributable to the ADVANCE-IT program; other factors are likely to have contributed.
Intervention Examined
NSF’s ADVANCE-IT Program
Features of the Intervention
Since 2001, the NSF has invested more than $130 million in support of ADVANCE programs nationwide. ADVANCE programs aim to increase the participation and advancement of women in academic science and engineering careers. The ADVANCE-IT program, in particular, consists of five-year, institution-wide programs that aim to transform recruitment policies and practices, as well as the climates and cultures, of colleges and universities to advance women in STEM positions.
Features of the Study
The authors compared the number of female faculty at the assistant professor rank in colleges of engineering before and after the ADVANCE-IT grant period to measure the impacts of targeted recruitment practices developed under the grant. Specifically, the study focused on changes in the number of full-time, tenure-track assistant professors from the start to the end of the grant period (2001–2006) and through the three years following the grant period (2006–2009). Associate and full professor ranks were excluded on the basis that any increase in senior female faculty was likely driven by promotions rather than recruitment. The study evaluated seven first-round recipients of ADVANCE-IT grants that offered engineering degrees from 2001 to 2009: Georgia Institute of Technology; New Mexico State University; University of California, Irvine; University of Colorado at Boulder; University of Michigan, Ann Arbor; University of Washington; and University of Wisconsin, Madison.
In addition to examining the net gain or loss of female faculty during the study period, the authors analyzed each institution’s recruitment practices and programs to identify similarities and differences in efforts among those with larger or smaller absolute gains in the number of female faculty. Data on female full-time, tenure-track engineering faculty at the assistant professor rank were obtained from the Profiles of Engineering and Engineering Technology Colleges published by the American Society for Engineering Education. Information on ADVANCE-related recruitment programs was obtained through grant proposals and final reports submitted by the institutions to the NSF and through the ADVANCE portal website.
Findings
- The study found no statistically significant relationships between participation in the ADVANCE-IT program and the number of female assistant professors at those institutions.
Considerations for Interpreting the Findings
The authors compared the outcomes of participating institutions measured before and after they participated in the ADVANCE-IT program. CLEAR’s guidelines require that the authors observe outcomes for multiple periods before the intervention to rule out the possibility that participating institutions had increasing or decreasing trends in the outcomes examined before enrollment in the program. That is, if institutions that had increasing numbers of female assistant professors tended to enroll in the program, we would anticipate further increases over time, even if they did not participate in the program. Without knowing the trends before program enrollment, we cannot rule this out.
Further, the authors did not conduct any tests of statistical significance to demonstrate that observed changes in the number of female assistant professors were meaningful and not due to chance. Lastly, none of the analyses controlled for institution-level characteristics such as average age and demographic composition, as required by CLEAR.
Causal Evidence Rating
The quality of causal evidence presented in this study is low because the authors conducted no tests of statistical significance and did not observe outcomes for multiple periods before the intervention. This means we are not confident that the estimated effects would have been attributable to the ADVANCE-IT program; other factors are likely to have contributed.