Citation
Sullivan, K., & Davis, R. (2007). Increasing retention of women engineering students. American Society for Engineering Education. Proceedings of the 2007 American Society for Engineering Education Annual Conference and Meeting. Washington, DC: American Society for Engineering Education.
Highlights
- The study’s objective was to examine how participating in a study-designed program called the Jornada Intervention affected undergraduate female engineering students’ attitudes toward and persistence in studying engineering. Program activities included receiving a handheld computer and an invitation to participate in meetings to share experiences using the computer and suggestions for improving the user’s experience.
- The study used a comparison group design to compare outcomes for the treatment group with those of women who enrolled in engineering in adjacent years. Data sources included an attitudinal survey administered for the first analysis and administrative records to measure students’ graduation rates for the second analysis.
- In the first analysis, the study found no significant differences in attitudes toward engineering between the treatment and comparison groups. For the second analysis, though results were not presented and methods were not described, the authors reported that they uncovered significant differences in persistence between the treatment and comparison groups. They noted that program participants were significantly more likely to graduate in engineering than both (1) women from the same year who did not participate in the program and (2) women who enrolled in engineering in adjacent years.
- The quality of causal evidence presented in this report is low because it does not include control variables to adjust for differences between treatment and comparison groups. This means we are not confident that the estimated effects are attributable to the intervention; other factors are likely to have contributed.
Intervention Examined
Jornada Intervention
Features of the Intervention
The study took place at Santa Clara University and included all female freshman and sophomore students enrolled in engineering in fall 1999 who agreed to participate in the Jornada Intervention (65 percent of those asked). Students in the intervention or treatment group received a handheld computer called the Hewlett-Packard Jornada, either as a personal digital assistant device or a 3/4-sized laptop computer, and were invited to attend regular meetings with other treatment students to evaluate the computer and discuss how to improve its functioning. Participants also completed a workshop that focused on improving future generations of cell phone technology. The purpose of the intervention was to (1) “increas[e] their self-identification as ‘techies’” and (2) form a group of female students who would meet to discuss technology and technological issues.
Features of the Study
The study included 54 participants in the treatment group and 17 students in the comparison group, comprising a single cohort, for the first analysis. Sample sizes were not provided for the second analysis, which included six comparison years.
For the first analysis, the authors administered a study-designed survey two years following implementation of the intervention using a scale constructed from attitudinal measures in the “Success in Engineering” instrument. Survey results were compared for the treatment and comparison groups and analyzed with a paired-samples t-test.
For the second analysis, the authors compared graduation rates for female students who entered the university during the year the intervention was implemented with those who enrolled in adjacent years, though they did not describe their methods for this analysis.
Findings
- The first analysis found no significant differences between the two-year follow-up attitudinal measures of the treatment and comparison groups.
- In the second analysis, the six-year graduation rate of the treatment group was 75.9 percent, compared with the 52.9 percent rate of the comparison group. The authors reported that these results were significant, though results were not presented and methods were not described.
Considerations for Interpreting the Findings
There are several factors to consider when interpreting this study. The study used a quasi-experimental approach to evaluating the intervention; however no control variables were used to adjust for differences between groups. Further, the composition of the two groups included some systematic variation: the treatment group contained freshmen and sophomores, whereas the control group contained only freshmen. Sophomores had already persisted from the first to the second year of college, making them perhaps more likely to graduate and have positive attitudes toward engineering.
Further, selection bias likely influenced the study outcomes, as students opted in to the intervention. Expressing interest in attending group meetings to discuss technology (one component of the intervention) is likely correlated with an outcome of interest—holding positive attitudes toward engineering. Selection into the study might also be correlated with a number of other factors, including motivation, work and family obligations, and collaborative working style, that are in turn correlated with persistence outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because it did not include control variables to adjust for differences between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to the intervention; other factors are likely to have contributed.