Title,Citation,Topic_area,Study_type,Study_evidence_rating,Outcome_effectiveness,Findings,Intervention_program,Topics,Target_population,Firm_characteristics,Geographic_setting,Original_publication_date,Original_publication_link,"Review Protocol" "Effect of a professional coaching intervention on the well-being and distress of physicians: A pilot randomized clinical trial","Dyrbye, L. N., Shanafelt, T. D., Gill, P. R., Satele, D. V., & West, C. P. (2019). Effect of a professional coaching intervention on the well-being and distress of physicians: A pilot randomized clinical trial. JAMA Internal Medicine, 179(10), 1406–1414.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes Health and safety-Mod/high-Favorable impacts Health and safety","Summary: The study’s objective was to examine the impact of professional coaching on physicians’ well-being outcomes.  The study is a randomized controlled trial. Participants completed surveys at the beginning and end of the five-month study period. The authors compared the outcomes of treatment and comparison group members.  The study found that participants who received professional coaching had a significant reduction in overall symptoms of burnout.   The quality of the causal evidence is high for health and safety outcomes related to burnout. This means we are confident that the estimated effects are attributable to professional coaching and not to other factors. The quality of causal evidence is low for health and safety outcomes related to quality of life and resilience, for attitudes outcomes related to job satisfaction, and for engagement and meaning at work because the authors did not include information on sample attrition nor did they account for preexisting differences between treatment and control groups. This means we are not confident that the estimated effects are attributable to professional coaching; other factors are likely to have contributed. ","Professional Coaching for Physicians",Mentoring,"Health professional",,,2019,https://pubmed.ncbi.nlm.nih.gov/31380892/,"Review Protocol" "Mindfulness training improves employee well-being: A randomized controlled trial","Slutsky, J., Chin, B., Raye, J., & Creswell, J. D. (2019). Mindfulness training improves employee well-being: A randomized controlled trial. Journal of Occupational Health Psychology, 24(1), 139–149. https://doi.org/10.1037/ocp0000132",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of a mindfulness training intervention on productivity and attitudes about attentional focus, job satisfaction, and work–life conflicts.   The study used a randomized controlled trial design. The authors used statistical models to compare the outcomes of treatment and control group members. Outcome data were collected through daily diaries that participants kept for three days, as well as through experience sampling surveys that participants received on their smartphones throughout the day, both before and after the intervention.  The study found the mindfulness intervention had a significant positive impact on attentional focus and a significant negative impact on work–life conflict.   The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the mindfulness training intervention, and not to other factors. ","Mindfulness Training Intervention","Other health and safety",,,,2019,https://doi.apa.org/doiLanding?doi=10.1037/ocp0000132,"Review Protocol" "Impact of a 5-week collegiate-level residential STEM summer program on secondary school students (Research to practice)","Campbell, B., Robb, S., Abbott, S., & Mutunga, S. (2014). Impact of a 5-week collegiate-level residential STEM summer program on secondary school students (Research to practice). Proceedings of the 2014 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of participating in the Pennsylvania Governor’s School for the Sciences’ (PGSS) five-week residential program in summer 2013 on high school seniors’ attitudes toward and interest in science, technology, engineering, and math (STEM). Using surveys developed for the study, the authors compared the outcomes of students before and after participating in the PGSS program. The study found that female students had a significantly improved disposition toward technology and general science and a significantly higher interest in the specific fields of chemistry, computer science, and math at the end of the program, compared to the start of the program. The quality of causal evidence presented in this report is low because the authors did not observe outcomes for multiple periods before the intervention nor account for selection into the intervention. This means we are not confident that the estimated effects are attributable to the PGSS program; other factors are likely to have contributed.","the Pennsylvania Governor’s School for the Sciences (PGSS)","Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2014,https://peer.asee.org/impact-of-a-5-week-collegiate-level-residential-stem-summ…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "An outcome evaluation of an adult education and postsecondary alignment program: The Accelerate New Mexico experience","Rivera, M., Davis, M., Feldman, A., & Rachkowski, C. (2013). An outcome evaluation of an adult education and postsecondary alignment program: The Accelerate New Mexico experience. Problems and Perspectives in Management, 11(4), 105-120.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of the Accelerate New Mexico Math Camp on outcomes for community college students, including women and minority students traditionally underrepresented in STEM. The study used course-administered assessments to compare the math competency and attitudes about math of students before and after participation in the Accelerate summer program. The authors found that students (both male and female) who participated in the Accelerate program had higher math competency and less anxiety about math after completing the program, compared with their competency and anxiety measured before enrollment in the program. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the Accelerate New Mexico Math Camp program; other factors are likely to have contributed.","the Accelerate New Mexico Math Camp","Science, Technology, Engineering, and Math (STEM) programs Community college education and other classroom training Youth programs","Youth, Other barriers, Female",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns","Smith, J., Lewis, K., Hawthorne, L., & Hodges, S. (2013). When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns. Personality & Social Psychology Bulletin, 39(2), 131-143. [Study 2]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Unfavorable impacts Attitudes","Summary: The study’s objective was to examine the impact of male-dominated marketing of a fictitious eco-psychology program on female students’ interest in pursuing the program. Authors randomly assigned 49 female students in an introductory psychology course to receive a brochure about a fictitious one-year master’s program in eco-psychology that was presented as either male-dominated or gender-neutral. Students were then asked to rate their motivation to enroll in the program. The study found that female students who read the male-dominated brochure displayed statistically significantly lower interest in pursuing the program than those in the gender-neutral group. The quality of causal evidence presented in this report is low because the authors did not include sufficient controls in their analysis to account for differences between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to type of brochure reviewed; other factors are likely to have contributed.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns","Smith, J., Lewis, K., Hawthorne, L., & Hodges, S. (2013). When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns. Personality & Social Psychology Bulletin, 39(2), 131-143. [Study 3]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study examined the effect of providing information about effort requirements to succeed in a fictitious male-dominated eco-psychology master’s program on female students’ attitudes toward their ability to succeed in this field. The authors randomly assigned 48 female undergraduates to receive one of four verbal messages from an advisor about their ability to succeed in a fictitious master’s program in eco-psychology; the women also received a printed brochure distributed to all study groups. The messages varied in terms of emphasis placed on students’ ability versus level of effort. Specifically, three of the message groups conveyed that student success depended on (1) their natural ability (natural ability group), (2) putting in more effort than peers (more effort group), or (3) putting in the same high level of effort as peers (normal effort group). The advisors did not discuss effort requirements with the fourth group that served as a control group (the no information group). Students then immediately completed a questionnaire measuring self-doubt, perceived confidence, future interest in eco-psychology, and how well they thought they would fit into the program. The study found that students in the normal effort group exhibited a significantly higher sense of academic belonging to and future interest in the eco-psychology program than the other three groups. They also had higher perceived competence compared with the more effort group and exhibited lower self-doubt than the no information and more effort groups. The quality of causal evidence presented in this report is low because the authors did not include sufficient controls in their analysis to account for differences between each group, which is required of randomized controlled trials (RCTs) with high attrition. This means we are not confident that the estimated effects are attributable to the type of message the study participants received; other factors are likely to have contributed.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Expanding Your Horizons: The impact of a one-day STEM conference on middle school girls’ and parents’ attitude toward STEM careers","Massi, L., Reilly, C., Johnson, D., & Castner, L. (2012). Expanding Your Horizons: The impact of a one-day STEM conference on middle school girls’ and parents’ attitude toward STEM careers. Proceedings of the 2012 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts Attitudes","Summary: The study’s objective was to estimate the effects of a one-day annual conference on middle school girls’ (grades 6–8) and their parents’ interest in and knowledge of science, technology, engineering, and mathematics (STEM). The goal of the conference, which was held at a large metropolitan, public university in central Florida, was to encourage young girls to consider STEM careers. The authors administered a single questionnaire to conference attendees that asked about attendees’ knowledge of and interest in STEM before and after the conference. The survey was included in backpacks distributed at the beginning of the day, and participants were reminded to complete it at the end of the day. Of the 214 middle school girls and 103 parent attendees, 80 percent of the girls and 37 percent of the parents turned in a completed survey. The authors did not conduct any statistical tests to measure changes in knowledge of and interest in STEM before and after participating in the conference. The quality of causal evidence presented in this report is low because the authors measured pre-intervention knowledge of and interest in STEM after participation in the intervention, which may be subject to recall bias. This means we are not confident that any estimated effects would be attributable to the Expanding Your Horizons (EYH) conference, but other factors are likely to have contributed. Further, the authors did not conduct statistical analysis to measure whether observed changes in outcomes were actually significant.","the Expanding Your Horizons (EYH) Conference","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2012,https://www.asee.org/public/conferences/8/papers/3204/download,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Inspiring girls and their female after school educators to pursue computer science and other STEM careers","Koch, M., & Gorges, T. (2012). Inspiring girls and their female after school educators to pursue computer science and other STEM careers. International Journal of Gender, Science and Technology, 4(3), 294-312.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts Attitudes","Summary: The study’s objective was to examine the impact of facilitating Build IT, an extracurricular information technology (IT) program for middle-school girls, on the facilitators’ interest in STEM education and careers. The authors also described outcomes for girls who participated in the facilitator-led program. The study relied on a survey administered to 31 program facilitators after they had been involved with the program for one year. The study found no statistically significant relationships between facilitating Build IT and interest in STEM education and careers. The quality of causal evidence presented in this report is low because the authors did not use a research design that supported causal inference or control for factors other than Build IT that could contribute to the observed outcomes.","Build IT","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Other barriers, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end","Shu, L., Mazar, N., Gino, F., Ariely, D., & Bazerman, M. (2012). Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proceedings of the National Academy of Sciences, 109(38), 15197-15200.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: The study’s objective was to examine whether placing a signature line at the beginning of a form, rather than at the end, which is more common, discouraged dishonesty. The authors conducted three separate randomized experiments. In experiments 1 and 2, individuals completed math puzzles and were asked to report the number of puzzles they completed, as well as their expenses for traveling to the site where the experiment was conducted. In experiment 3, individuals acquiring a new car insurance policy were asked to report their current cars’ odometer readings on an insurance form. In each experiment, individuals were randomly assigned either to receive a form requiring a signature at the top, or to receive a form requiring a signature at the bottom. The study found that, across the three experiments, participants who signed at the top of the form were less likely to respond dishonestly than participants who signed the form at the bottom or did not sign the form at all. The quality of causal evidence presented in this report is high because it was based on three well-implemented randomized controlled trials. This means we are confident that the estimated effects are attributable to the placement of the signature line on the forms, and not to other factors.",,"Behavioral Interventions",Adult,,"United States",2012,,"Behavioral Insights Review Protocol" "The effects of hands-on activities on middle school females’ spatial skills and interest in engineering and technology-based careers","Phelps, M. (2012). The effects of hands-on activities on middle school females’ spatial skills and interest in engineering and technology-based careers. Proceedings of the 2012 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of participating in technology-based three-dimensional hands-on activities on middle school girls’ interest in becoming an engineer or technologist. The activities took place at a 2006 Society of Women Engineers-sponsored event and included designing and constructing a vehicle and assembling an electrical circuit. The author conducted statistical analyses to compare self-reported career interests of girls who participated in the hands-on activities and those who did not, as measured by a program-administered survey. The study found that participation in the hands-on activities was positively related to increased interest in becoming an engineer or technologist. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the hands-on activities; other factors are likely to have contributed.","the Intervention","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female","Professional, scientific, and technical services","United States",2012,https://peer.asee.org/the-effects-of-hands-on-activities-on-middle-school-femal…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Inquiry-based science and technology enrichment program: Green earth enhanced with inquiry and technology","Kim, H. (2011). Inquiry-based science and technology enrichment program: Green earth enhanced with inquiry and technology. Journal of Science Education & Technology, 20(6), 803-814.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of the Inquiry-based Science and Technology Enrichment Program (InSTEP)—a week-long, half-day summer science curriculum for female 8th-grade students—on participants’ interest and confidence in studying science. The author used a survey to compare students’ attitudes about science before and after participating in InSTEP. The author found that among the female 8th-grade students who chose to enroll in InSTEP, interest in science increased, gender stereotypes about science and scientists were undermined, and overall attitudes toward science improved by the end of the program. There were no significant differences in students’ anxiety about taking science courses. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to InSTEP; other factors are likely to have contributed.","Inquiry-based Science and Technology Enrichment Program (InSTEP)","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "The role of living–learning programs in women’s plans to attend graduate school in STEM fields","Szelényi, K., & Inkelas, K. (2011). The role of living–learning programs in women’s plans to attend graduate school in STEM fields. Research in Higher Education, 52(4), 349-369.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to assess the impact of participating in a female-only STEM living-learning program on undergraduate female STEM majors’ intentions to pursue graduate studies in a STEM field. The authors conducted regression analysis using data from the 2004–2007 National Study of Living Learning Programs (NSLLP) to determine if women who participated in female-only STEM living-learning programs were more likely than women in other living arrangements to express plans to attend graduate school in a STEM field. The study found that the female undergraduate STEM majors living in a female-only STEM living-learning community were more likely to have plans to attend graduate school in a STEM field than female STEM majors living in coeducational STEM living-learning communities, non-STEM living-learning communities, or traditional residence halls. The quality of causal evidence presented in this report is low because the authors’ analysis did not account for important individual characteristics related to the outcome that could differ across groups. This means we are not confident that the estimated effects are attributable to female-only STEM living-learning programs. Other factors are likely to have contributed.","STEM Living-Learning Programs","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Increasing saving behavior through age-progressed renderings of the future self","Hershfield, H., Goldstein, D., Sharpe, W., Fox, J., Yeykelis, L., Carstensen, L., & Bailenson, J. (2011). Increasing saving behavior through age-progressed renderings of the future self. Journal of Marketing Research, 48(SPL), S23–S37.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: The study’s objective was to determine whether contact with an age-progressed visual representation of the self led participants to allocate more money to savings for retirement. The authors presented four random assignment studies relying on varying degrees of contact with these age-progressed visual representations in an attempt to pinpoint their effect on retirement savings behavior and the mechanism through which such representations alter savings attitudes. In the four experiments discussed here, the authors collected data directly from participants by administering post-intervention surveys or collecting hypothetical asset allocation information. The study found that participants who saw age-progressed pictures of themselves chose to allocate a significantly higher proportion of hypothetical income to retirement compared to a control group; the participants in this study were drawn from a group of adults ages 18 to 35. Results were similar in three other experiments using undergraduate students. The quality of the causal evidence presented in this study is high. This means we are confident the differences in outcomes observed between treated and control groups resulted from the interventions tested, and not other factors.","Interactions with Age-Processed Visual Representation of the Self","Employer provided retirement benefits",Youth,,"United States",2011,,"Behavioral Finance: Retirement Review Protocol" "STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM)","Stout, J., Dasgupta, N., Hunsinger, M., & McManus, M. (2011). STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100, 255-270. [one of three studies described in a single report]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of having a professor and teaching assistant (TA) of the same gender on introductory calculus students’ attitudes toward math, compared with English. The study used computerized tasks to measure attitudes toward math and collected the expected course grade and actual course performance of female students with a female professor and TA pair (a female teaching pair), compared with those of female students with a male professor and TA pair (a male teaching pair) at both the start and end of the semester. The study found that female students with a female teaching pair identified more with math and expected a higher course grade than those with a male teaching pair. Female students with a male teaching pair had more negative attitudes toward math, compared with English, than female students with a female teaching pair. The quality of causal evidence presented in this report is low because the authors did not demonstrate comparability of treatment and control groups before the intervention. This means we are not confident that the estimated effects are attributable to having a female teaching pair; other factors are likely to have contributed.",,"Science, Technology, Engineering, and Math (STEM) programs Youth programs",Youth,,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM)","Stout, J., Dasgupta, N., Hunsinger, M., & McManus, M. (2011). STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100, 255-270. [one of three studies described in a single report]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Low-Unfavorable impacts Attitudes","Summary: The study’s objective was to examine the impact of reading a paragraph-long biography of a female engineer on female engineering majors’ implicit and explicit attitudes toward math and English, their self-efficacy in engineering, their identification with the engineers in the assigned text, and their intention to pursue a career in engineering. The study used computerized tasks to measure outcomes for female students who read a paragraph-long biography of a female engineer compared with those of female students who read either (1) biographies of male engineers or (2) a description of engineering innovations with no mention of the engineer’s gender. The study found that attitudes of female students who read the female biography did not favor math over English or vice versa, whereas those who read the male biography or the description of engineering innovations demonstrated negative implicit attitudes toward math and a preference for English over math. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to reading a paragraph-long biography about female engineers, and not to other factors.",,"Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Do female and male role models who embody STEM stereotypes hinder women’s anticipated success in STEM?","Cheryan, S., Siy, J., Vichayapai, M., Drury, B. & Kim, S. (2011). Do female and male role models who embody STEM stereotypes hinder women’s anticipated success in STEM? Social Psychological and Personality Science, 2(6), 656-664.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of interacting with an upper-level college role model on female students’ beliefs that they could succeed as a computer science major. The study used a randomized controlled trial to assign female students majoring in a discipline other than computer science to meet with one of four different types of upper-level students who varied by gender and whether they embodied perceived stereotypes of computer science majors. The authors administered a questionnaire to estimate impacts of the different intervention conditions. The study found that women who interacted with a stereotypical computer science role model believed they would be less successful as a computer science major compared with women who interacted with a nonstereotypical computer science role model and with women who did not interact with a role model. The quality of causal evidence provided in this study is low because the study had high attrition and the authors did not demonstrate baseline equivalence between the samples or include adequate controls to ensure the resulting groups were similar on all relevant characteristics. This means we are not confident that the effects estimated in this study are attributable to role model interaction; other factors are likely to have contributed.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM)","Stout, J., Dasgupta, N., Hunsinger, M., & McManus, M. (2011). STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100, 255-270. [one of three studies described in a single report]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of interacting with a male or female upperclass student majoring in math and psychology (hereafter referred to as a male or female peer expert) on female undergraduate STEM majors’ attitudes toward math and performance on a math test. The study used computerized tasks and students’ self-reported attitudes to compare performance in and attitudes toward math of female undergraduate students who interacted with a female peer expert and those who interacted with a male peer expert. The study found that female students who interacted with a male peer expert exhibited negative implicit attitudes toward math compared with English, whereas female students who interacted with a female peer expert had the same attitudes toward math and English. Female students who interacted with a female peer expert attempted more problems on the math test than those who interacted with a male peer expert. The quality of causal evidence presented in this report is high, as it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to interacting with a female peer expert, and not to other factors.",,"Science, Technology, Engineering, and Math (STEM) programs Science, Technology, Engineering, and Math (STEM) programs Youth programs",Youth,,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Misunderstanding savings growth: Implications for retirement savings behavior","McKenzie, C., & Liersch, M. (2011). Misunderstanding savings growth: Implications for retirement savings behavior. Journal of Marketing Research, 68, S1–S13.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: Experiments Using Undergraduate Students This study used three experiments in a controlled setting to examine undergraduate students’ intuitions of retirement savings growth, understanding of investment concepts, and motivation to invest toward their retirement after entering the workforce.1 The authors administered three surveys consisting of hypothetical situations followed by exercises asking respondents to compute an omitted element of the savings function (for example, the monthly deposit amount or rate of return required to accumulate a given amount of savings at retirement). Each experiment randomly assigned participants to receive or not receive some form of aid (for example, a calculator) or additional information (for example, savings growth charts) to complete the exercises. The first study found that participants tended to severely underestimate projected savings account balances at retirement and overestimate the rates of return and monthly deposit amounts required to ensure such balances. The second study found that when provided with information on exponential growth, participants were more motivated to save for retirement; however, in the third study there was no statistically significant relationship between information on exponential growth and motivation to save for retirement. The quality of the causal evidence presented in this portion of the study is high. This means we are confident that any differences in responses were a result of the various treatment conditions of the three experiments. Analysis of Employees’ Survey Data The study further aimed to determine whether showing employees how much money they would have at retirement—based on current annual contributions, the current account balance, and return rates—can motivate them to increase savings, compared with showing employees current account balances only. The authors analyzed differences in employees’ interest in changing current levels of retirement savings, measured on a seven-point Likert scale, comparing those who were provided only their estimated 401(k) account balance at retirement and those who were shown only their current balance. The analysis demonstrated that showing workers actual account projections (in addition to the current balance) motivated them to save more. The quality of causal evidence presented in this portion of the study is also high. This means we are confident that showing employees their estimated 401(k) account balances at retirement increased the percentage of employees who wanted to save more each month.","Informational Interventions for Undergraduates and Workers","Employer provided retirement benefits","Adult, Employed",,"United States",2011,,"Behavioral Finance: Retirement Review Protocol" "Choice proliferation, simplicity seeking, and asset allocation","Iyengar, S., & Kamenica, E. (2010). Choice proliferation, simplicity seeking, and asset allocation. Journal of Public Economics, 94, 530-539.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: Laboratory experiments This study used two laboratory experiments to examine the relationship between the number of choices offered and decision making.  Individuals were asked to choose between risky gambles and sure payoffs, with the number of choices randomly determined. The authors found that when individuals were presented with a larger number of choices, they tended to choose simpler payoff options. The quality of the causal evidence presented in this portion of the study is high. This means we are confident that an increase in the number of choices an individual is given causes him or her to choose a less complex payoff option. Analysis of 401(k) data  The study further aimed to determine whether the number of fund options offered by a 401(k) plan influenced the share of assets allocated to a given class of funds (for example, equities or bonds) by plan members. The authors analyzed differences in portfolio allocations based on the number of funds available using regression analysis, controlling for other 401(k), firm, and employee characteristics. The analysis demonstrated that as the number of investment options increased, people tended to allocate more of their 401(k) savings to money market and bond funds at the expense of equity funds. The quality of causal evidence presented in this portion of the study is low. This means we cannot be confident that an increase in the number of funds offered in a retirement plan causes an increase in the share of funds allocated to money market and bond funds and a concurrent decrease in the share of funds allocated to equities.","Changing the Number of Risky Choices","Employer provided retirement benefits",Employed,,"United States",2010,,"Behavioral Finance: Retirement Review Protocol" "How and to what extent does entrepreneurship education make students more entrepreneurial? A California case of the Technology Management Program","Tam, H. (2009). How and to what extent does entrepreneurship education make students more entrepreneurial? A California case of the Technology Management Program (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 304852166).","Entrepreneurship and Self-Employment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of participation in the University of California, Santa Barbara’s Technology Management Program (TMP) on students’ entrepreneurial attitudes. The author compared the entrepreneurial attitudes of students at the beginning and the end of the spring 2008 academic term. The study found that the overall entrepreneurial attitudes of students participating in the TMP became stronger over the course of the term. The quality of causal evidence presented in this report is low because the author did not account for pre-intervention trends in participants’ entrepreneurial attitudes. This means we are not confident that the estimated effects are attributable to the TMP courses the students took; other factors are likely to have contributed.","the Technology Management Program (TMP)","Other training and education",Adult,,"United States",2009,,"Employment and Training Review Protocol" "Examining the impact of small business institute participation on entrepreneurial attitudes","Harris, M., Gibson, S., & Taylor, S. (2008). Examining the impact of small business institute participation on entrepreneurial attitudes. Journal of Small Business Strategy, 18(2), 57-75.","Entrepreneurship and Self-Employment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of attending an undergraduate class as part of the Small Business Institute (SBI), an entrepreneurial program, on students’ entrepreneurial attitudes. The authors compared students’ entrepreneurial attitudes before and after taking an SBI class in one of six universities across the United States. Entrepreneurial attitudes were measured using the Entrepreneurial Attitudes Orientation survey. The study found a positive relationship between participation in an SBI class and students’ entrepreneurial attitudes. The quality of causal evidence presented in this report is low because the authors compared the attitudes of students measured at a single point before and after they participated in the class, but did not compare these changes in attitudes with those of a comparison group of students who did not take an SBI class. This means we are not confident that the estimated effects are attributable to the SBI class; other factors are likely to have contributed.","SBI Courses","Other training and education",Adult,"Small business","United States",2008,,"Employment and Training Review Protocol" "Effects of learning about gender discrimination on adolescent girls’ attitudes toward and interest in science","Weisgram, E., & Bigler, R. (2007). Effects of learning about gender discrimination on adolescent girls’ attitudes toward and interest in science. Psychology of Women Quarterly, 31, 262-269.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: The study’s objective was to examine the impact of learning about gender discrimination in the sciences on girls’ attitudes toward and interest in science. The study included middle school girls who attended a conference designed to improve their attitudes toward and interest in science. Participants were randomly assigned to the treatment condition, which included standard conference sessions on science careers and a session on gender discrimination in the sciences, or to the control group that attended only the standard sessions. The authors administered pre- and post-conference surveys to measure the impact of the intervention. The study found significant increases in reported science self-efficacy and science utility value among the girls in the treatment condition, whereas girls in the control condition experienced a decrease in egalitarian attitudes toward science. The quality of causal evidence provided in this study is high because it was a well-implemented randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the intervention and not to other factors.",,"Preventing discrimination Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2007,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Increasing retention of women engineering students","Sullivan, K., & Davis, R. (2007). Increasing retention of women engineering students. American Society for Engineering Education. Proceedings of the 2007 American Society for Engineering Education Annual Conference and Meeting. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts Attitudes Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine how participating in a study-designed program called the Jornada Intervention affected undergraduate female engineering students’ attitudes toward and persistence in studying engineering. Program activities included receiving a handheld computer and an invitation to participate in meetings to share experiences using the computer and suggestions for improving the user’s experience. The study used a comparison group design to compare outcomes for the treatment group with those of women who enrolled in engineering in adjacent years. Data sources included an attitudinal survey administered for the first analysis and administrative records to measure students’ graduation rates for the second analysis. In the first analysis, the study found no significant differences in attitudes toward engineering between the treatment and comparison groups. For the second analysis, though results were not presented and methods were not described, the authors reported that they uncovered significant differences in persistence between the treatment and comparison groups. They noted that program participants were significantly more likely to graduate in engineering than both (1) women from the same year who did not participate in the program and (2) women who enrolled in engineering in adjacent years. The quality of causal evidence presented in this report is low because it does not include control variables to adjust for differences between treatment and comparison groups. This means we are not confident that the estimated effects are attributable to the intervention; other factors are likely to have contributed.","Jornada Intervention","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2007,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Assessing the impact of small business training on nascent entrepreneurs in Illinois","Harfst, K. (2006). Assessing the impact of small business training on nascent entrepreneurs in Illinois (Doctoral dissertation). Retrieved from ProQuest Dissertation and These database (UMI No. 3229827).","Entrepreneurship and Self-Employment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts Attitudes","Summary: The study’s objective was to examine the impact of participation in a Small Business Development Center (SBDC) seminar on intentions to start a business. The author administered a survey of his own design before and after respondents participated in an SBDC seminar and compared the respondents’ pre- and post-seminar responses on a series of items measuring intentions to start a business. The study found no statistically significant relationships between participation in the SBDC seminar and intentions to start a business. The quality of causal evidence presented in this report is low because the author did not account for pre-intervention trends in participants’ entrepreneurial intentions. This means that we would not be confident that any estimated effects would be attributable to the SBDC seminar; however, the study found no effects.",,"Other training and education",Adult,"Small business","United States",2006,http://www.illinois.gov/dceo/SmallBizAssistance/CenterConnect/Strategic%20Plan/…,"Employment and Training Review Protocol" "4 schools for WIE","Erkut, S., Marx, F., & Wellesley College Center for Research on Women. (2005). 4 schools for WIE. Evaluation report. Wellesley, MA: Wellesley Centers for Women.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Mixed impacts Attitudes","Summary: The study’s objective was to determine the effect of the 4 Schools for Women in Engineering (WIE) strategy on 8th-grade students’ attitudes toward STEM careers. The strategy involved forming teams of mostly female engineering professors, students, and practitioners to train middle school teachers on engineering and gender-inclusive teaching methods and to serve as in-class role models for students. The authors administered a study-designed survey at the beginning and end of the school year to a single cohort of students to measure changes in outcomes following implementation of the intervention. The authors estimated program impacts using regression models and paired t-tests. The study found that students’ attitudes toward science improved after the intervention, but attitudes toward engineering worsened. There was no change in students’ attitudes toward math. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the 4 Schools for WIE strategy; other factors are likely to have contributed.","4 Schools for Women in Engineering","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2005,http://files.eric.ed.gov/fulltext/ED500754.pdf,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Do scientists help people? Beliefs about scientists and the influence of pro-social context on girls’ attitudes toward physics","Yanowitz, K. (2004). Do scientists help people? Beliefs about scientists and the influence of pro-social context on girls’ attitudes toward physics. Journal of Women and Minorities in Science and Engineering, 10(4), 393-399.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-No impacts Attitudes","Summary: The study’s objective was to examine the impact of reading a passage featuring a pro-social context on careers in physics on female students’ interest in science. The authors randomly assigned 88 5th- and 6th-grade girls to four conditions. In the treatment condition, participants read a story describing a female physicist who entered that profession to benefit society. Participants assigned to the control conditions read different stories. After reading these stories, the participants responded to a short survey asking how much they liked the story and how interested they were in becoming physicists. The study found no relationship between the treatment story and participants’ interest in becoming physicists. The quality of causal evidence presented in this study is high because it was based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects would be attributable to the scientists’ stories and not to other factors. However, the study did not find statistically significant effects.",,"Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2004,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"