Title,Citation,Topic_area,Study_type,Study_evidence_rating,Outcome_effectiveness,Findings,Intervention_program,Topics,Target_population,Firm_characteristics,Geographic_setting,Original_publication_date,Original_publication_link,"Review Protocol"
"Synthesis Report: Behavioral Finance Synthesis: Findings",,"Behavioral Finance: Retirement",,,,"Findings:

  
  
  

  
  
  

  
  
  

          People have relatively limited knowledge about saving for retirement and can be induced to save more when provided with additional information.

          Making retirement more salient, by having people think of themselves in retirement or providing a target retirement date, can increase intentions to save and alter investment choices.

          People can become overwhelmed by the number of investment options they face; when this occurs, they tend to use simple rules to make decisions.",,"Employer provided retirement benefits","Adult, Older worker",,"United States",,,"Behavioral Finance: Retirement Review Protocol"
"Integrating economic strengthening and family coaching to reduce work-related health hazards among children of poor households: Burkina Faso","Karimli, L., Rost, L., & Ismayilova, L. (2018). Integrating economic strengthening and family coaching to reduce work-related health hazards among children of poor households: Burkina Faso. Journal of Adolescent Health, 62, S6-S14.","Child Labor","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Child labor-Low-Favorable impacts
      
    
  
              


      
            Child labor
      


  
      
            Health and safety-Low-No impacts
      
    
  
              


      
            Health and safety","Summary:

The study’s objective was to examine the impact of the Trickle Up and Trickle Up Plus programs on children’s exposure to work-related hazards and abuse, and children’s work-related health outcomes. This summary focuses on the comparison between the Trickle Up Plus intervention group and the control group.
The study was a randomized controlled trial in the Nord Region of Burkina Faso, West Africa. Using three waves of survey data administered to children and adults separately, the authors conducted statistical models to compare the outcomes of treatment and control group members.
The study found that Trickle Up Plus program participation was significantly related to a reduction of hazardous work and abuse, compared with the control group. However, the study found no statistically significant relationship between the Trickle Up program and work-related health outcomes.
The quality of causal evidence presented in this report is low because it was a randomized controlled trial with unknown attrition and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Trickle Up Plus program; other factors are likely to have contributed.","Trickle Up Plus Program","Child labor Health","Female, Other, Low income",,International,2018,https://www.researchgate.net/publication/322086777_Integrating_Economic_Strengt…,"Child Labor Review Protocol"
"Integrating economic strengthening and family coaching to reduce work-related health hazards among children of poor households: Burkina Faso","Karimli, L., Rost, L., & Ismayilova, L. (2018). Integrating economic strengthening and family coaching to reduce work-related health hazards among children of poor households: Burkina Faso. Journal of Adolescent Health, 62, S6-S14.","Child Labor","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Child labor-Low-Favorable impacts
      
    
  
              


      
            Child labor
      


  
      
            Health and safety-Low-No impacts
      
    
  
              


      
            Health and safety","Summary:

The study’s objective was to examine the impact of the Trickle Up and Trickle Up Plus programs on children’s exposure to work-related hazards and abuse, and children’s work-related health outcomes. This summary focuses on the comparison between the Trickle Up intervention group and the control group.
The study was a randomized controlled trial in the Nord Region of Burkina Faso, West Africa. Using three waves of survey data administered to children and adults separately, the authors conducted statistical models to compare the outcomes of treatment and control group members.
The study found that Trickle Up program participation was significantly related to a reduction of hazardous work and abuse, compared with the control group. However, the study found no statistically significant relationship between the Trickle Up program and work-related health outcomes.
The quality of causal evidence presented in this report is low because it was a randomized controlled trial with unknown attrition and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Trickle Up program; other factors are likely to have contributed.","Trickle Up Program","Child labor Health","Female, Other, Low income",,International,2018,https://www.researchgate.net/publication/322086777_Integrating_Economic_Strengt…,"Child Labor Review Protocol"
"The impact of Medicare Part D on self-employment.","Moulton, J. G., Diebold, J. C., & Scott, J. C. (2017). The impact of Medicare Part D on self-employment. Research on Aging, 39(1), 64-85.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the effect of the introduction of Medicare Part D on self-employment. The authors used a nonexperimental approach and data from the American Community Survey to estimate the impact.
The authors found that self-employment significantly increased by 0.5 percentage points after the program began.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects on self-employment are attributable to the introduction of Medicare Part D, although other factors might also have contributed.","Medicare Part D","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2017,,"Employment and Training Review Protocol"
"Does protecting older workers from discrimination make it harder to get hired? Evidence from disability discrimination laws.","Neumark, D., Song, J., & Button, P. (2017). Does protecting older workers from discrimination make it harder to get hired? Evidence from disability discrimination laws. Research on Aging, 39(1), 29-63.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the strength of state disability discrimination laws on hiring rates of older men.
The study used statistical models and data from the Health and Retirement Study and the Survey of Income and Program Participation to estimate the impacts.
The study found no relationship between the strength of disability discrimination laws and the hiring rate for older unemployed disabled and nondisabled men.
The quality of casual evidence presented in this report is low because the study is a nonexperimental analysis that did not demonstrate that the groups being compared were similar and did not account for possible differences in the analysis. This means we are not confident that the estimated effects are attributable to state disability discrimination laws; other factors are likely to have contributed.","Disability discrimination laws","Health Older workers' programs","Adult, Older worker, Male",,"United States",2017,https://journals.sagepub.com/doi/full/10.1177/0164027516656142,"Employment and Training Review Protocol"
"The effect of the Affordable Care Act on labor transitions among older workers.","Camilleri, S. E. (2017). The effect of the Affordable Care Act on labor transitions among older workers. In Three papers on the Affordable Care Act (Doctoral dissertation). North Carolina State University, Raleigh, NC.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-No impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the Affordable Care Act (ACA) on the employment outcomes of older workers.
The author used data from the Current Population Survey for 2011–2016 and regression analyses to estimate impacts.
The study found that the ACA was not associated with employment outcomes among older workers.
The quality of causal evidence presented in this report is moderate because the study is based on a well-implemented nonexperimental design. This means we are somewhat confident that any estimated effects are attributable to the ACA, but other factors might also have contributed.","Affordable Care Act (ACA)","Older workers' programs Health insurance","Adult, Older worker",,"United States",2017,,"Employment and Training Review Protocol"
"Job insecurity, unemployment insurance and on-the-job search. Evidence from older American workers.","Gutierrez, I. A. (2016). Job insecurity, unemployment insurance and on-the-job search. Evidence from older American workers. Labour Economics, 41, 228-245. doi:10.1016/j.labeco.2016.05.011","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the potential wage replacement rate through Unemployment Insurance (UI) on the probability of transitioning to non-employment.
The study used a nonexperimental design and data from the Health and Retirement Study (HRS) from 1996–2006 and 2010–2012 to identify workers who expected job loss during the baseline survey and who experienced job transitions by the follow-up survey two years later.
The study found no statistically significant relationships between potential wage replacement rates and transitions to non-employment.
The quality of causal evidence presented in this report is low because the analyses do not sufficiently account for other factors that might contribute to the estimated effects. This means we are not confident that the estimated effects are attributable to the potential wage replacement rate through UI; other factors are likely to have contributed.","Unemployment Insurance (UI)","Older workers' programs Unemployment Insurance","Adult, Older worker, Employed",,"United States",2016,,"Employment and Training Review Protocol"
"Eliminating child labour in El Salvador through economic empowerment and social inclusion: Impact report","de Hoop, J., Kovrova, I., & Rosati, F. C. (2016). Eliminating child labour in El Salvador through economic empowerment and social inclusion: Impact report. Retrieved from http://www.ucw-project.org/attachment/07032017224El_Salvador_IE_07042016_web.pdf","Child Labor","Study Type: Descriptive Analysis",,,"Summary:

	The study’s objective was to assess the effects of a women’s entrepreneurship training program on children’s participation in school and work.
	The study was a regression discontinuity design, using baseline and follow-up household surveys to measure program outcomes. Households were selected into the program if they had a wealth score below a cut-off point. The authors compared the outcomes of children in households above and below the cut-off score to understand the program’s effects.
	The study found that selection and participation in the program was significantly associated with a decrease in the percent of children working-only and not attending school, and the number of hours that children worked per week. Selection and participation in the program was also significantly related to an increase in school attendance.
	This study used a regression discontinuity design and therefore was reviewed using CLEAR’s descriptive study evidence review guidelines. As such, it does not receive a causal rating.","Eliminating Child Labour in El Salvador through Economic Empowerment and Social Inclusion","Child labor","Female, Parent, Low income",,International,2016,http://www.ucw-project.org/attachment/07032017224El_Salvador_IE_07042016_web.pdf,"Child Labor Review Protocol"
"Spousal labor market effects from government health insurance: Evidence from a veterans affairs expansion.","Boyle, M. A. & Lahey, J. N. (2016). Spousal labor market effects from government health insurance: Evidence from a veterans affairs expansion. Journal of Health Economics, Elsevier, 45(C), 63-76.","Older Workers, Veterans","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Mod/high-No impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the expansion of health insurance for veterans through the U.S. Department of Veterans Affairs (VA) in the mid-1990s (as a result of the implementation of the Veteran’s Health Care Eligibility Reform Act) on the employment and earnings of non-covered spouses.
The authors used a statistical model and data from the Census Bureau’s March Current Population Survey and the Health and Retirement Study to compare the employment and earnings of wives of male veterans with wives of non-veterans of the same ages before and after the expansion of health insurance for veterans through the VA.
The study found no statistically significant associations between the expansion of health insurance for veterans and the employment of their wives and mixed evidence on associations between the expansion of health insurance for veterans and the earnings of their wives.
The quality of causal evidence presented in this report is moderate for the employment outcomes because it was based on a well-implemented nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to the expansion of health insurance for veterans through the VA, but other factors might also have contributed. The quality of causal evidence presented in this report is low for earnings outcomes because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the expansion of health insurance for veterans through the VA; other factors are likely to have contributed.","Veteran Health Insurance Expansion","Health Older workers' programs Veterans' reemployment","Adult, Older worker, Veteran or military",,"United States",2016,,"Older Workers, Veterans Review Protocol"
"Nursing interventions to help prevent children from working on the streets","Mert, K., & Kadioglu, H. (2016). Nursing interventions to help prevent children from working on the streets. International Nursing Review 63, 429–436. doi:10.1111/inr.12301","Child Labor","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Child labor-Low-No impacts
      
    
  
              


      
            Child labor","Summary:

The study’s objective was to assess the impact of a program developed to prevent children under 18 years of age from working on the street.
The study was an interrupted time series design conducted in one neighborhood in the city of Izmit, Turkey. The authors compared the outcomes for mothers and their children before, immediately after, and three months after participating in The Streets are Not the Solution program.
The study found that participating in the program was associated with a 44 percentage point decrease from pretest to posttest in the percentage of children who worked on the streets. However, the study did not include tests of statistical significance.
The quality of causal evidence presented in this report is low because the authors did not account for trends in outcomes before the intervention. This means we are not confident that the effects are attributable to the Streets are Not the Solution program; other factors are likely to have contributed.","The Streets are Not the Solution","Child labor","Female, Other, Parent",,International,2016,,"Child Labor Review Protocol"
"The effect of Social Security information on the labor supply and savings of older Americans.","Armour, P., & Lovenheim, M. F. (2016). The effect of Social Security information on the labor supply and savings of older Americans. (Working paper no. 361). Ann Arbor, MI: University of Michigan, Michigan Retirement Research Center.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Mixed impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the Social Security statements issued by the Social Security Administration (SSA) on older men’s annual hours worked.
The authors used a nonexperimental design and data from Health and Retirement Study surveys and Social Security administrative records to estimate the impacts.
The study found that receiving the first Social Security Statement is associated with a decrease in self-reported annual hours worked and receiving the second Social Security Statement is associated with an increase in self-reported annual hours worked.
The quality of casual evidence presented in this report is moderate because that the study was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to Social Security statements, but other factors might also have contributed.","Social Security statements","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2016,,"Employment and Training Review Protocol"
"Men and women expecting to work longer: Do changing work conditions matter?","Moen, P., Kojola, E., Kelly, E.L., & Karakaya, Y. (2016). Men and women expecting to work longer: Do changing work conditions matter? Work, Aging and Retirement, 2(3), 321-344.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of a work flexibility intervention called STAR (Support. Transform. Achieve. Results) on workers’ expectations of retirement age and bridge jobs.
The study is a randomized controlled trial that uses surveys of workers before the intervention and five years post-intervention.
The study found that receiving STAR was associated with higher expectations of working at age 65 and retiring at age 67 or older.
The quality of causal evidence presented in this report is low because, although it was based on a randomized controlled trial, it had unknown attrition and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to STAR; other factors are likely to have contributed.","STAR (Support. Transform. Achieve. Results.)","Work based and other occupational training Other employer services","Adult, Older worker",,"United States",2016,https://experts.umn.edu/en/publications/men-and-women-expecting-to-work-longer-…,"Employment and Training Review Protocol"
"The effect of health reform on retirement.","Levy, H., Buchmueller, T. C., & Nikpay, S. (2015). The effect of health reform on retirement (Research Paper 329). Ann Arbor, MI: Michigan Retirement Research Center.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of the expansion of Medicaid to low-income adults in 2014 on retirement rates of workers ages 55 to 64.
The authors used a nonexperimental analysis to compare those who lived in an expansion state to those who lived in other states. The study used data from the monthly Current Population Survey from January 2005 through July 2015.
The study found no statistically significant relationships between Medicaid expansion and retirement.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the expansion of Medicaid; other factors are likely to have contributed.","Medicaid expansion","Health insurance Older workers' programs","Adult, Older worker, Low income",,"United States",2015,https://deepblue.lib.umich.edu/handle/2027.42/116062,"Employment and Training Review Protocol"
"The impact of a randomly assigned time and place management initiative on work and retirement expectations.","Cahill, K. E., James, J. B., & Pitt-Catsouphes, M. (2015). The impact of a randomly assigned time and place management initiative on work and retirement expectations. Work, Aging and Retirement, 1(4), 350-368.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

	The study examined the impact of a time and place management (TPM) initiative at a medical provider on retirement expectations among workers ages 50 and older.
	The study was a randomized control trial and used a series of surveys and administrative data to estimate the impact.
	The study found that, when surveyed 12 months after randomization, employees in the treatment group were statistically significantly more likely to expect to directly exit their job as opposed to staying in their current job within the next five years.
	The quality of causal evidence presented in this report is moderate because the study was a randomized controlled trial with high attrition, but the authors did demonstrate that the treatment and control groups were similar before the intervention. This means we are somewhat confident that the estimated effects are attributable to TPM, but other factors might also have contributed.","Time and Placement Management (TPM)","Older workers' programs","Older worker","Health care and social assistance",,2015,https://hrs.isr.umich.edu/publications/biblio/8231,"Employment and Training Review Protocol"
"Employment outcomes among African American and white women with disabilities: Examining the inequalities.","Balcazar, F. E., & Suarez-Balcazar, Y. (2015). Employment outcomes among African American and white women with disabilities: Examining the inequalities. Women, Gender, and Families of Color, 3(2), 144-164.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of adult vocational rehabilitation (VR) services on employment outcomes for African American and white women.
The authors used a statistical model and data from a Midwestern state VR system to compare the odds of employment after completing VR services for older versus younger women.
The study found no significant difference in employment outcomes between older and younger women who completed VR services.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before receiving VR services. This means we are not confident that the estimated effects are attributable to participation in VR services; other factors are likely to have contributed.","Vocational Rehabilitation (VR) Services","Vocational rehabilitation","Adult, Older worker, Female, Black or African American, White, Disability",,"United States",2015,,"Employment and Training Review Protocol"
"The effect of a severe health shock on work behavior: Evidence from different health care regimes.","Datta Gupta, N., Kleinjans, K. J., & Larsen, M. (2015). The effect of a severe health shock on work behavior: Evidence from different health care regimes. Social Science and Medicine, 136-137, 44-51.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of the national health policies of the United States and Denmark on employment of older workers after a health shock, defined as a new cancer or cardiovascular disease diagnosis.
The authors used a nonexperimental approach to compare the impact of a health shock on the likelihood of nonemployment in the United States’ private multipayer system with Denmark’s nationalized universal health care model. The authors used the data from the RAND Health and Retirement Study survey from 1994 to 2002 and Danish population registers for 1993 to 2001.
The study found no differences between the United States and Denmark in the relationship between a health shock and employment for older workers.
The quality of causal evidence presented in this report is low because the authors did not account for unobserved differences between the countries. This means we are not confident that differences in the association between health shocks and employment in the United States and Denmark are attributable to the national health care policies; other factors are likely to have contributed.","National Healthcare Policy","Health Older workers' programs","Adult, Older worker",,"International, United States",2015,https://www.sciencedirect.com/science/article/pii/S0277953615002816,"Employment and Training Review Protocol"
"How much does access to health insurance influence the timing of retirement?","Coe, N. B., & Goda, G. S. (2015). How much does access to health insurance influence the timing of retirement? (SIEPR Discussion Paper No. 14-007). Stanford, CA: Stanford Institute for Economic Policy Research. City, ST: Publisher.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-No impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined the impact of state-level reforms of health insurance on early retirement and Social Security retirement benefit claims.
The authors used 1996–2010 data from the Health and Retirement Study (HRS) and statistical models to compare the outcomes of individuals in states with state-level reforms of health insurance with those in non-reform states.
The study found that employed individuals in states with non-group health insurance reform were significantly more likely to retire at age 63.
The quality of causal evidence presented in this report is low because the authors did not demonstrate that individuals in states with state-level reforms of health insurance were similar to those in non-reform states. This means we are not confident that the estimated effects are attributable to the non-group health insurance reform; other factors are likely to have contributed.","Retiree Health Insurance","Health insurance Older workers' programs","Adult, Older worker",,"United States",2015,,"Employment and Training Review Protocol"
"Who pays the high health costs of older workers? Evidence from prostate cancer screening mandates.","Bailey, J. (2014). Who pays the high health costs of older workers? Evidence from prostate cancer screening mandates. Applied Economics, 46(32), 3931-3941. doi:10.1080/00036846.2014.948673","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Mod/high-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of state mandates that health insurance plans cover prostate cancer screenings on the earnings and employment of men older than age 50
The study used nonexperimental regression analyses to compare older men in states that had enacted a mandate with women, younger men, and older men living in states without a mandate. The author used data from the Current Population Survey(CPS) between 1990 and 2009.
The study found that older men in states with mandates had lower wages and were less likely than the comparison group to be employed.
The quality of causal evidence presented in this study is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to the prostrate screening mandate, but other factors might also have contributed.","Prostate cancer screening mandates","Health insurance Older workers' programs","Adult, Older worker, Male",,"United States",2014,https://sites.temple.edu/jamesbailey/files/2013/08/Who-Pays-the-High-Health-Cos…,"Employment and Training Review Protocol"
"The labor-market returns to community college degrees, diplomas, and certificates","Jepsen, C., Troske, K., & Coomes, P. (2014). The labor-market returns to community college degrees, diplomas, and certificates. Journal of Labor Economics, 32(1), 95-121.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine the effect of earning community college degrees, diplomas, and certificates on students’ later earnings.
Using administrative data from the Kentucky Community and Technical College System (KCTCS), the National Student Clearinghouse, and Kentucky’s unemployment insurance program, the authors compared the outcomes of students who completed college degrees, diplomas, and certificates with those of students who did not.
The study found that community college degrees, diplomas, and certificates improved earnings for those who received them, compared with those who did not receive them but were otherwise similar.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to earning community college degrees, diplomas, and certificates, but other factors might also have contributed.","the Program Under Study","Other training and education","Adult, Female, Male",,"United States",2014,,"Review Protocol"
"Mentoring practices proven to broaden participation in STEM disciplines","Crumpton-Young, L., Elde, A., & Ambrose, K. (2014, June). Mentoring practices proven to broaden participation in STEM disciplines. Paper presented at 2014 American Society for Engineering Education Annual Conference, Indianapolis, IN.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study identified effective mentoring practices, mostly at postsecondary institutions, for women and minority students in science, technology, engineering, and mathematics (STEM) fields.
	The authors surveyed 25 randomly selected mentor programs or individuals who received a National Science Foundation award for being successful in mentoring students who are typically underrepresented (including women) in STEM. The survey asked award recipients to rate their performance on 22 mentoring practices using a scale ranging from not doing well to doing exceptionally well. The authors deemed those mentoring practices rated most highly as best practices.
	The study found that the highest-rated mentoring practices were being passionate about mentees and their development, creating opportunities for mentees, setting high expectations for mentees’ performance, providing needed support to mentees, and respecting mentees’ confidentiality. Conversely, the lowest-rated practices set the mentor in an authoritative light and included providing constant feedback to and having constant contact with mentees.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Black or African American, Asian, Hispanic of any race, Multiracial, American Indian or Alaska Native, Native Hawaiian or other Pacific Islander",,"United States",2014,https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwiM7t…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Training program impacts and the onset of the Great Recession. [Adult sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [Adult sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine whether participation in the Workforce Investment Act (WIA) Adult Program services increased participants' earnings and if so, whether those gains were greater during a recessionary period than in periods before and after the recession.
The authors analyzed administrative data to compare the quarterly earnings of participants receiving services through the WIA Adult Program with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year.
The study found that males and females receiving services from the WIA Adult Program from June to December 2007 (before the recession) earned more, on average, than the comparison group in some, but not all, of the 16 follow-up quarters examined. Males and females receiving services from July 2008 to June 2009 and from July 2009 to June 2010 (after the onset of the recession) tended to earn less, on average, than the comparison group in most of the 16 follow-up quarters.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Adult Program services, but other factors might also have contributed.
The report also examined the impacts of WIA Dislocated Worker services; that profile is available here.","the Workforce Investment Act (WIA) Adult Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.missourieconomy.org/pdfs/WDQI_MissouriReport_TrainingProgramImpacts…,"Review Protocol"
"Community college men and women: A test of three widely held beliefs about who pursues computer science","Denner, J., Werner, L., O’Connor, L., & Glassman, J. (2014). Community college men and women: A test of three widely held beliefs about who pursues computer science. Community College Review, 42(4), 342-362.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study's objective was to examine factors that affect motivation to pursue a computer and information sciences (CIS) degree, with the intention of informing strategies to increase the number of community college students, particularly females, who go on to enroll in a CIS degree program at a four-year university. Specific factors examined were programming and/or video-gaming experience; support from peers, instructors, mentors, and family members; and level and type of motivation to pursue a CIS degree.
	The authors administered a series of three (baseline and 6- and 18-month follow-up) surveys to students enrolled in an introductory programming class at one of 15 California community colleges and ran multiple linear regression models by gender to determine each of the three factors’ influence on the student’s plan to pursue a four-year CIS degree, comparing results for women versus men.
	The study found that female students, who were overall less likely to intend to pursue a four-year CIS degree, reported receiving significantly less encouragement from their peers to persist in computer science and spending less time playing computer games than male students. Predictors for women choosing to pursue a four-year CIS degree changed over time: strong immediate (baseline) predictors were interest in computer science, video gaming, and peer encouragement; a year after baseline, the strongest predictors were expectations for success and interest in solving problems.",,"Youth programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2014,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Impact of a 5-week collegiate-level residential STEM summer program on secondary school students (Research to practice)","Campbell, B., Robb, S., Abbott, S., & Mutunga, S. (2014). Impact of a 5-week collegiate-level residential STEM summer program on secondary school students (Research to practice). Proceedings of the 2014 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:

	
	
		The study’s objective was to examine the impact of participating in the Pennsylvania Governor’s School for the Sciences’ (PGSS) five-week residential program in summer 2013 on high school seniors’ attitudes toward and interest in science, technology, engineering, and math (STEM).
		Using surveys developed for the study, the authors compared the outcomes of students before and after participating in the PGSS program.
		The study found that female students had a significantly improved disposition toward technology and general science and a significantly higher interest in the specific fields of chemistry, computer science, and math at the end of the program, compared to the start of the program.
		The quality of causal evidence presented in this report is low because the authors did not observe outcomes for multiple periods before the intervention nor account for selection into the intervention. This means we are not confident that the estimated effects are attributable to the PGSS program; other factors are likely to have contributed.","the Pennsylvania Governor’s School for the Sciences (PGSS)","Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2014,https://peer.asee.org/impact-of-a-5-week-collegiate-level-residential-stem-summ…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The role of intensive case management services in reentry: The northern Kentucky female offender reentry project","McDonald, D., & Arlinghaus, S. L. (2014). The role of intensive case management services in reentry: The northern Kentucky female offender reentry project. Women & Criminal Justice, 24(3), 229-251.",Reentry,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Mixed impacts
      
    
  
              


      
            Education and skills gains
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

	This study examined the impact of the Northern Kentucky Female Offender Reentry Project’s intensive case management (ICM) services on releasees’ education, employment, and recidivism outcomes.
	Using program data and administrative data from state and county judicial databases, the study compared outcomes of the treatment group, which received ICM services pre-release (in prison) and post-release (in the community), with outcomes of the comparison group, which only received ICM services pre-release (in prison).
	The study found mixed results for education, employment, and recidivism. The authors did not conduct statistical significance tests.
	The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Female Offender Reentry Project’s pre- and post- release ICM services; other factors are likely to have contributed.","Northern Kentucky Female Offender Reentry Project’s intensive case management (ICM) services","Other employment and reemployment Reentry","Justice-involved, Female",,"United States",2014,https://doi.org/10.1080/08974454.2014.909759,"Employment and Training Review Protocol"
"Exploring gender imbalance among STEM doctoral degree recipients (Issue brief)","Gillen, A., & Tanenbaum, C. (2014, September). Exploring gender imbalance among STEM doctoral degree recipients (Issue brief). Washington, DC: American Institutes for Research.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study’s main objective was to accurately identify the nature of gender imbalances in the receipt of science, technology, engineering, and mathematics (STEM) degrees, specifically isolating imbalances at the doctoral level.
	Using data from the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS), the authors calculated the ratio of women to men earning doctoral degrees by field, controlling for the number who completed an undergraduate degree in the same field (called “prepared” students).
	The study found that, overall, men are overrepresented in 76 percent of doctoral fields. This means that, after controlling for the number of students who complete an undergraduate degree in a given field, more men than women are completing doctoral degrees in the same field. However, there is slightly less gender imbalance of prepared students earning doctoral degrees in all STEM fields than in non-STEM fields, even though the absolute number of men receiving STEM doctoral degrees is higher.
	The authors suggest that because female undergraduate STEM majors persist from undergraduate to graduate levels at almost the same rate (90 percent as often) as men, gender imbalance must be addressed earlier in the academic pipeline to address overall disparities in the receipt of STEM doctoral degrees.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, Male, STEM professional",,"United States",2014,http://www.air.org/sites/default/files/downloads/report/STEM%20PhDs%20Gender%20…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Training program impacts and the onset of the Great Recession. [TAA sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [TAA sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine whether participation in the Trade Adjustment Assistance (TAA) Program services increased participants’ earnings and, if so, whether those gains were greater during a recessionary period than in periods before and after the recession.
The authors analyzed administrative data to compare the quarterly earnings of TAA participants with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year.
The study found that males and females who received services through the TAA Program earned less, on average, than the comparison group during almost all of the 16 follow-up quarters examined. This held regardless of whether the participants began receiving services before, during, or after the recession.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to services received through the TAA Program, but other factors might also have contributed.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other training and education","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.empiwifo.uni-freiburg.de/sonstige-dateien/heinrich-mueser_trainingi…,"Review Protocol"
"Did age discrimination protections help older workers weather the Great Recession?","Neumark, D., & Button, P. (2014). Did age discrimination protections help older workers weather the Great Recession? Journal of Policy Analysis and Management, 33(4), 566-601. doi:10.1002/pam.21762","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined whether the strong age discrimination laws moderated the impact of the Great Recession on employment outcomes of older workers.
The study used statistical models and the data from the 2003–2011 Current Population Survey (CPS) and 2004–2011 Quarterly Workforce Indicators to estimate impacts.
The study found that states with stronger age discrimination laws had experienced increases in unemployment rates, longer unemployment durations, and decreases in hiring rates for some types of older workers relative to younger workers in the months during and following the Great Recession.
The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the differential effect of the Great Recession on older workers in states with strong age discrimination laws, but other factors might also have contributed.","Age Discrimination Laws","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Female, Male",,"United States",2014,https://www.nber.org/papers/w19216,"Employment and Training Review Protocol"
"Training program impacts and the onset of the Great Recession. [Dislocated worker sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [Dislocated worker sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine whether participation in the Workforce Investment Act (WIA) Dislocated Worker Program services increased participants' earnings and if so, whether those gains were greater during a recessionary period than in periods before and after the recession.
The authors analyzed administrative data to compare the quarterly earnings of participants receiving services through the WIA Dislocated Worker Program with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year.
The study found that males receiving services through the WIA Dislocated Worker Program from July 2007 to December 2007 (before the recession), from January 2008 to June 2008 (at the beginning of the recession), and from July 2008 to June 2009 (during the worst of the recession) earned less than males in the comparison group in most of the 16 follow-up quarters examined. In no quarter did females participating in the WIA Dislocated Worker Program before, during, or after the recession earn more than females in the comparison group.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Dislocated Worker Program services, but other factors might also have contributed.
The report also examined the impacts of WIA Adult Program services; that profile is available here.","the Workforce Investment Act (WIA) Dislocated Worker Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs Job search assistance and supportive services","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.missourieconomy.org/pdfs/WDQI_MissouriReport_TrainingProgramImpacts…,"Review Protocol"
"The impact of earnings disregards on the behavior of low‐income families.","Matsudaira, J. D., & Blank, R. M. (2014). The impact of earnings disregards on the behavior of low‐income families. Journal of Policy Analysis and Management, 33(1), 7-35.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Low-No impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-No impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-No impacts
      
    
  
              


      
            Public benefit receipt","Summary:

	The study’s objective was to examine the impact of earnings disregards—the amount of earned income that is not counted when calculating whether someone is eligible for public benefits—for welfare assistance on the employment, earnings, and public assistance receipt of low-skilled single mothers.
	The study used a difference-in-differences approach to compare the outcomes of women before and after changes in the disregard rate within their state relative to women in other states over the same time period.
	The study did not find any statistically significant effects of earnings disregards on the outcomes of interest.
	The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to earnings disregards, but other factors might also have contributed. However, the study did not find statistically significant effects.","Earnings Disregards for Welfare Assistance","Other employment and reemployment","Female, Parent, Low income",,"United States",2014,,"Employment and Training Review Protocol"
"Distinctive and unique outreach programs: Promoting academic excellence and diversity","Monaco, P., & Morse, A. (2014). Distinctive and unique outreach programs: Promoting academic excellence and diversity. Proceedings of the 2014 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study examined a series of residential environmental engineering summer camps run by a southern technology university that were intended to stimulate interest in STEM fields, primarily for girls and underrepresented minority groups. Three different types of summer camps were offered for different populations: international mixed-gender Brazilian students, female-only, and local mixed-gender students.
	The authors reviewed qualitative data collected from 81 4th- through 12th-grade students participating in one of five summer camp sessions. These data included pre-session oral surveys; ongoing observations of class participation; and post-session group discussions, activity worksheets that asked for open-ended application of STEM theories to examples, and oral surveys.
	According to post-session discussions and surveys, across the five camp sessions, about 65 percent of camp participants, of whom 25 percent were females, expressed interest in pursuing a STEM major. The authors also reported that the females had higher levels of participation and engagement in female-only sessions compared with mixed-gender sessions.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2014,https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The economics of federal tax and employment laws: Empirical evidence from three studies.","Figinski, T. F. (2013). The economics of federal tax and employment laws: Empirical evidence from three studies. (Doctoral dissertation). Retrieved from ProQuest. Accession No. 1415455735.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-Unfavorable impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined the impact of removing the Social Security earnings test on Social Security claims, earnings, and labor force participation for female beneficiaries.
The study used a statistical model and the data from the Social Security Administration’s Benefit and Earnings Public Use File to estimate impacts.
The study found that removing the earnings test was associated with an increase in Social Security claims among women who were primary or spousal beneficiaries and with an increase in average earnings and in labor force participation among women who were primary beneficiaries.
The qaulity of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the change in Social Security earings test. This means we are not confident that the estimated effects are attributable to the removal of the Social Security earnings test; other factors are likely to have contributed.","Social Security 2000 Earnings Test Removal","Federal retirement benefits Older workers' programs","Adult, Older worker, Female",,"United States",2014,,"Employment and Training Review Protocol"
"Retiree health insurance for public school employees: Does it affect retirement?","Fitzpatrick, M. D. (2014). Retiree health insurance for public school employees: Does it affect retirement? Journal of Health Economics, 38, 88-98.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of an Illinois Public Schools retiree health insurance program on the retirement rates of eligible staff ages 55 to 75.
The study used a nonexperimental approach to compare the retirement rates of eligible staff with those of staff who were not eligible and staff in years prior to the introduction of the health insurance program who would have been eligible. The author used data from the Illinois Public Schools' Teacher Service Record database from 1970 to 1992.
The study found that eligibility for the program was associated with an overall decrease in retirement age, with eligible staff more likely to retire at ages 55 or 56, but less likely to retire after age 65 compared with those who were not eligible for the program.
The quality of the causal evidence presented in this report is low because the study is a nonexperimental analysis that does not account for possible differences in the groups. This means we are not confident that the estimated effects are attributable to the Illinois Public Schools Teachers Retirement Health Insurance Plan; other factors are likely to have contributed.","Illinois Public Schools Teachers Retirement Health Insurance Plan","Employer provided retirement benefits Health insurance Older workers' programs","Adult, Older worker",,"United States",2014,https://www.nber.org/papers/w19524,"Employment and Training Review Protocol"
"Women in community colleges: Access to success","St. Rose, A., & Hill, C. (2013). Women in community colleges: Access to success. Washington, DC: American Association of University Women.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The authors described two programs partnering with California community colleges—the Mathematics, Engineering, Science Achievement Community College Program (MCCP) and the CalWomen Tech Program—that aim to encourage women to pursue careers in science, technology, engineering, and mathematics (STEM). The two initiatives approach this goal in different ways: MCCP prepares women to transfer from two- to four-year STEM degree programs, whereas CalWomen Tech encourages participation in non-traditional career and technical education programs in STEM, such as information technology and automotive repair.
	MCCP offers a wide array of academic, career, and social support services, from academic tutoring and advising to a cohort-based course progression. CalWomen Tech’s promotional materials target women, such as “women in technology” websites that highlight the employment and salary opportunities available to women who choose non-traditional career paths, and skill-building or “bridge” courses that lay the groundwork for more advanced study.
	In 2010, 38 percent of MCCP transfer students from two- to four-year degree programs were women. Of those transferring to programs in life sciences or mathematics, more than 50 percent were women.
	After partnering with CalWomen Tech, female enrollment in City College of San Francisco’s computer networking and information technology program increased from 19 percent in 2006 to 33 percent in 2008, with persistent enrollment rates through 2010. Similarly, CalWomen Tech helped Evergreen Valley College raise female retention in its automotive technology program from 58 percent to 88 percent over the course of two years.",,"Youth programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2013,http://www.aauw.org/files/2013/05/women-in-community-colleges.pdf,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Perfectionism moderates stereotype threat effects on STEM majors’ academic performance","Rice, K., Lopez, F., Richardson, C., & Stinson, J. (2013). Perfectionism moderates stereotype threat effects on STEM majors’ academic performance. Journal of Counseling Psychology, 60(2), 287-293.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts
      
    
  
              


      
            Education and skills gains","Summary:


The study’s objective was to examine the impact of “stereotype threat priming” on students’ grades in postsecondary science, technology, engineering, and mathematics (STEM) courses and on self-reported feelings of self-efficacy in science. The authors argued that women and racial minorities perform worse in certain fields, such as STEM, when they feel threatened by stereotypes implying that their gender or race makes them inferior to their male or white peers. For this study, the authors primed treatment students to think about their gender, race, and ethnicity before completing a survey to measure self-efficacy in science. This priming was intended to make feelings of stereotype threat more acute.
The study randomly assigned students majoring in a STEM field to either the treatment group, which received stereotype threat priming before taking the survey, or the control group, which took the same survey but did not receive stereotype threat priming. The authors collected data from an author-generated online survey on feelings of self-efficacy and from academic records for course grades.
The study found that the stereotype threat priming had no significant effects by gender when comparing outcomes for the treatment and control groups.
The quality of causal evidence presented in this report is low because this is a randomized controlled trial with unknown attrition, and the authors did not control for background characteristics likely to affect the outcomes of interest. This means we are not confident that the estimated effects would have been attributable to stereotype threat priming; other factors are likely to have contributed.","the Evaluation of Stereotype Threat Priming","Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Other barriers, Female, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"An effective model for enhancing underrepresented minority participation and success in geoscience undergraduate research","Blake, R., Liou-Mark, J., & Chukuigwe, C. (2013). An effective model for enhancing underrepresented minority participation and success in geoscience undergraduate research. Journal of Geoscience Education, 61(4), 405-414.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		This study described the implementation of the National Science Foundation Research Experiences for Undergraduates (REU) program at New York City College of Technology. This year-long program aimed to retain undergraduates in STEM disciplines and prepare them for STEM careers, especially for underrepresented minority groups, including women.
		The authors, as directors of the REU program, described each of its components, including research, minicourses, seminars and workshops, mentoring, support and networking opportunities, and presenting at and attending conferences. Outcomes were discussed in terms of post-program experiences and achievements, as well as quotes from program participants.
		Overall, the authors concluded that the program was successful in terms of demonstrated interest in, attendance in, or completion of a STEM graduate program; achievement in STEM publications; and attainment of STEM awards. These outcomes were complemented by five quotes from student participants who discussed how the program changed their academic lives.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Attracting girls to civil engineering through hands-on activities that reveal the communal goals and values of the profession","Colvin, W., Lyden, S., & León de, l. B. (2013). Attracting girls to civil engineering through hands-on activities that reveal the communal goals and values of the profession. Leadership & Management in Engineering, 13(1), 35-41.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study determined the impact of a four-week series of civil engineering workshops on Australian middle school girls’ perceptions of engineers and the engineering profession. The goal of the workshops—which included hands-on activities and discussions of engineering theory, examples, and careers—was to familiarize girls with engineering and depict it as a profession in which communal goals are central, which research shows that women value more than men.
	The authors asked female students in grades 5 and 6 to draw and write about an engineer (the Draw an Engineer Test [DAET]) before and after they attended the workshops. The DAET was analyzed for the engineer’s characteristics, such as gender, work activity, and themes (for example, teamwork). The students also completed feedback forms at the end of the program.
	The authors found that by the end of the workshops, students were more likely to draw and describe engineers as women and happy as compared with the pre-program assessment in which the students often depicted engineers as car mechanics. Participants also referenced communal goals more frequently in the follow-up assessment.
	Participants reported in feedback forms a preference for hands-on activities over interactive discussions.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,International,2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The link between pensions and retirement timing: Lessons from California teachers.","Brown, K. M. (2013). The link between pensions and retirement timing: Lessons from California teachers. Journal of Public Economics, 98(C), 1-14.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of pension reform under the California State Teachers’ Retirement System (CalSTRS) on retirement timing.
The author used a statistical model and data from retirement-eligible employees in Los Angeles Unified School District from 1997 to 2000 to estimate the impact.
The study found public school employees were more likely to work an extra year in response to changes in pension plans that provided additional bonuses tied to age and tenure.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the changes in CalSTRS. This means we are not confident that the estimated effects are attributable to the changes in CalSTRS; other factors are likely to have contributed.","CalSTRS Defined Benefit Program","Older workers' programs Other wages and benefits","Adult, Older worker",,"United States",2013,https://eml.berkeley.edu//~saez/course/brown_jpube13.pdf,"Employment and Training Review Protocol"
"From the shop floor to the kitchen floor: Maternal occupational complexity and children’s reading and math skills","Yetis-Bayraktar, A., Budig, M., & Tomaskovic-Devey, D. (2013). From the shop floor to the kitchen floor: Maternal occupational complexity and children’s reading and math skills. Work and Occupations, 40(1), 37-64.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The authors sought to determine whether the complexity of a mother’s occupation (or the quality of employment as defined by its task complexity, autonomy, and authority) affected her 6- to 13-year-old children’s math and reading test scores. The study considered both the complexity of the mother’s occupation when her children were ages 6 to 13 as well as the complexity of her occupation in her children’s earliest years (birth to age 3).
	The authors analyzed data from the 1984–1996 Panel Study of Income Dynamics and its 1997 Child Development Supplement to determine the relationship between maternal occupational complexity and children’s scores on the Revised Woodcock-Johnson Test of Achievement. The analysis first accounted for traits associated with mothers’ employment status, that is, whether the mother was employed. Assuming the mother was employed, the authors then assessed the association between the complexity of the mother’s occupation and her children’s performance on the reading and math sections of the test.
	The study found that a one-point increase on the maternal white collar occupational complexity scale was associated with a 2.49-point increase in children’s test scores, whereas a one-point increase on the blue collar occupational complexity scale corresponded to a 1.01-point gain in test scores.
	Maternal occupational complexity during the child’s first three years also was associated with increased test scores later in life; a one-point gain in white collar occupational complexity in the first three years of the child’s life was associated with a 3.7-point increase in test scores when the child was ages 6 to 13.",,"Other employment and reemployment","Employed, Female, Parent",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The impact of uncapping of mandatory retirement on postsecondary institutions.","Weinberg, S. L., & Scott, M. A. (2013). The impact of uncapping of mandatory retirement on postsecondary institutions. Educational Researcher, 42(6), 338-348.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of uncapping mandatory retirement on retirement ages at postsecondary institutions
The study used a statistical model to examine the age of retirement before and after the policy change in 1993. It used data on faculty’s employment and retirement from a research university with multiple schools.
The study found that uncapping was associated with a statistically significant increase in the retirement age among those who retired from the university.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar nor did they control for differences between the groups. This means we are not confident that the estimated effects are attributable to the federal Age Discrimination in Employment Act (ADEA); other factors are likely to have contributed.","Age Discrimination Employment Act (ADEA)","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2013,https://www.researchgate.net/publication/258134776_The_Impact_of_Uncapping_of_M…,"Employment and Training Review Protocol"
"Not lack of ability but more choice: Individual and gender differences in choice of careers in science, technology, engineering, and mathematics","Wang, M., Eccles, J., & Kenny, S. (2013). Not lack of ability but more choice: Individual and gender differences in choice of careers in science, technology, engineering, and mathematics. Psychological Science, 24(5), 770-775.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This paper examined whether people with high math and verbal abilities were more likely to work in STEM occupations at the age of 33 compared with people with high math but only moderate verbal abilities. The authors also considered whether employment in STEM occupations varied by gender.
	The authors surveyed 1,490 people drawn from a national longitudinal study in the United States—once in 1992 when the respondents were in 12th grade and again in 2007 when the respondents were 33 years old—to ask about occupation type. They also collected math and verbal Scholastic Assessment Test (SAT) scores for these people, as well as the number of high school mathematics courses taken and parental reports of parental education and family income. The authors used regression analyses of longitudinal survey data and other descriptive statistical techniques.
	The study found that students’ high school math and verbal abilities predicted whether they worked in a STEM field at age 33. Of those with moderate verbal and high math abilities, 49 percent worked in a STEM job at age 33, whereas only 34 percent of those with both high verbal and math abilities worked in a STEM job. Though more women were in the high math and high verbal abilities group than men, the likelihood of someone with both high math and verbal abilities choosing to work in a STEM occupation did not vary by gender but did vary by the individual’s self-perception on abilities in math in 12th grade.
	The authors argued that people with high math and high verbal abilities could find a wider variety of jobs that fit their skill sets, which might explain why fewer of those in this group pursued STEM careers than those with high math but moderate verbal abilities.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, Male, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Motivational pathways to STEM career choices: Using expectancy–value perspective to understand individual and gender differences in STEM fields","Wang, M., & Degol, J. (2013). Motivational pathways to STEM career choices: Using expectancy–value perspective to understand individual and gender differences in STEM fields. Developmental Review, 33(4), 1-37","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This paper was a literature review of research about gender differences in STEM education levels and careers in the United States, including any evidence available for specific cultural, biological, and psychological theories for gender differences in STEM.
	The authors summarized evidence from existing studies, organized around three main types of influences on decision making: psychological (perceived self-competence, goals, and interests); socialization (societal and cultural beliefs that shape an individual’s belief and value systems); and biological (hormonal and genetic factors).
	The authors found evidence of psychological influences on women’s participation in STEM careers, including women preferring more people-oriented jobs than STEM careers typically provide, and women leaving more intensive STEM careers after having children because of their work–family preferences. In terms of socialization influences, the authors found evidence of the importance of school factors, including teachers’ instructional practices, the structure of schools or classrooms, and teachers’ abilities to forge positive relationships with students, in influencing female students’ interest and abilities in STEM fields. In addition, the study highlighted evidence that values, belief systems, and family support can influence girls’ interest and success in STEM fields.
	The study noted that other studies have found biological differences in terms of hormone production and brain organization between men and women, but that these differences have not conclusively been linked to any behavior or ability levels in men and women.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Accelerating change for women faculty of color in STEM: Policy, action, and collaboration","Institute for Women’s Policy Research. (2013). Accelerating change for women faculty of color in STEM: Policy, action, and collaboration. Washington, DC: Institute for Women’s Policy Research.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		This report highlighted a 2013 convening of stakeholders who discussed barriers to becoming successful STEM faculty for black, Hispanic, and Native American women, and strategies to overcome those barriers. The Institute for Women’s Policy Research organized the convening.
		The report was compiled from a series of conference sessions presented at the convening by about 50 experts from various academic, political, corporate, and community sectors. Session topics included the current status of women of color in STEM, areas of progress and nonprogress, and current initiatives to increase the representation of women of color in STEM faculty positions. Attendees provided recommendations for improvement.
		Conference representatives identified three main barriers to the advancement of minority female faculty: workplace climate policies that did not meet the needs of female faculty of color, a lack of multicultural perspectives, and a lack of embracement of diversity in academic departments. Health, financial, and family issues and obligations; high community service demands; and a lack of social support also created unique challenges for this population.
		Recommendations for improvement included raising awareness of underrepresentation of minority women in STEM faculty positions, monitoring and publicizing institutions’ progress on diversity in STEM faculty, and tailoring faculty mentoring programs or tools to effectively serve female faculty of color.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, STEM professional, Black or African American, Hispanic of any race, American Indian or Alaska Native","Professional, scientific, and technical services","United States",2013,http://www.iwpr.org/publications/pubs/accelerating-change-for-women-faculty-of-…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Why academic STEM mothers feel they have to work harder than others on the job","Kmec, J. (2013a). Why academic STEM mothers feel they have to work harder than others on the job. International Journal of Gender, Science, & Technology, 5(2), 80-101.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This report analyzed faculty members’ perceptions of how hard they have to work at their job based on their sex and parental status, specifically in the context of science, technology, engineering, and mathematics (STEM) disciplines that are typically characterized by a masculine work culture. The objective was to contribute to the knowledge base on gender issues in academic professions and the specific challenges that working mothers in STEM fields face.
	The study used data from the 2011 Faculty Caregiving and Workplace and Culture survey administered online to about 300 tenure-line faculty members in all disciplines at a large public U.S. university. The author conducted regression analyses to estimate the link between faculty members’ sex, parental status, academic discipline, and their level of agreement with the statement, “My job requires me to work very hard.” The regression controlled for several factors that might affect faculty members’ perceptions of how hard they have to work at their job, such as length of time at current institution, academic rank, perceived job demand levels, feelings of being valued, marital status, age of children, household/family responsibilities, and degree to which household responsibilities are perceived to affect job performance.
	The study found that mothers in STEM fields were more likely to strongly agree that they have to work very hard at their job, compared to fathers in both STEM and non-STEM fields and mothers in non-STEM fields, after controlling for a number of factors related to perceptions of effort required. Specifically, fathers in STEM and non-STEM fields were 84 percent and 92 percent less likely, respectively, to strongly agree that they have to work very hard at their job than mothers in STEM fields. Similarly, mothers in non-STEM fields were about 81 percent less likely than mothers in STEM fields to agree that they have to work very hard at their job.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, Male, STEM professional, Parent",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns","Smith, J., Lewis, K., Hawthorne, L., & Hodges, S. (2013). When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns. Personality & Social Psychology Bulletin, 39(2), 131-143. [Study 3]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:


The study examined the effect of providing information about effort requirements to succeed in a fictitious male-dominated eco-psychology master’s program on female students’ attitudes toward their ability to succeed in this field.
The authors randomly assigned 48 female undergraduates to receive one of four verbal messages from an advisor about their ability to succeed in a fictitious master’s program in eco-psychology; the women also received a printed brochure distributed to all study groups. The messages varied in terms of emphasis placed on students’ ability versus level of effort. Specifically, three of the message groups conveyed that student success depended on (1) their natural ability (natural ability group), (2) putting in more effort than peers (more effort group), or (3) putting in the same high level of effort as peers (normal effort group). The advisors did not discuss effort requirements with the fourth group that served as a control group (the no information group). Students then immediately completed a questionnaire measuring self-doubt, perceived confidence, future interest in eco-psychology, and how well they thought they would fit into the program.
The study found that students in the normal effort group exhibited a significantly higher sense of academic belonging to and future interest in the eco-psychology program than the other three groups. They also had higher perceived competence compared with the more effort group and exhibited lower self-doubt than the no information and more effort groups.
The quality of causal evidence presented in this report is low because the authors did not include sufficient controls in their analysis to account for differences between each group, which is required of randomized controlled trials (RCTs) with high attrition. This means we are not confident that the estimated effects are attributable to the type of message the study participants received; other factors are likely to have contributed.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Training and retirement patterns.","Montizaan, R., Cörvers, F., & de Grip, A. (2013). Training and retirement patterns. Applied Economics, 45(15), 1991-1999. doi:10.1080/00036846.2011.646066","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of firm-specific training and general training on older men’s retirement decisions
The study used a nonexperimental design and data from the U.S. National Longitudinal Survey of Older Men for 1966 to 1983 to estimate impacts.
The study found that workers with firm-specific training were more likely to retire within the time period studied and to have jobs with a mandatory retirement age than workers without firm-specific training.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before receiving training and did not account for self-selection into training. This means we are not confident that the estimated effects are attributable to firm-specific or general training; other factors are likely to have contributed.","Firm-specific & General Training","Work based and other occupational training","Adult, Older worker, Male",,"United States",2013,https://www.researchgate.net/publication/236021967_Training_and_retirement_patt…,"Employment and Training Review Protocol"
"Does retiree health insurance encourage early retirement?","Nyce, S., Schieber, S. Shoven, J. B., Slavov, S., & Wise, D. A. (2013). Does retiree health insurance encourage early retirement? Journal of Public Economics, 104, 40-51.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of employer-provided retiree health insurance on early retirement.
The authors used a nonexperimental design to compare early retirement outcomes of employees who are eligible for retiree health insurance with that of those who have no coverage.
The study found that people who were employed at age 57 and eligible for employer-subsidized retiree health coverage were significantly more likely to retire at ages 62 to 64 (before becoming eligible for Medicare at age 65) compared with those with no coverage.
The quality of causal evidence presented in this report is low because the authors did not account for other factors that could have affected the difference between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to retiree health insurance; other factors are likely to have contributed.","Retiree Health Insurance","Employer provided retirement benefits Older workers' programs","Adult, Older worker",,"United States",2013,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3770310/,"Employment and Training Review Protocol"
"Gender equity issues in CTE and STEM education","Toglia, T. (2013). Gender equity issues in CTE and STEM education. TechDirections, 72, 14-18.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This article summarized current gender equity issues in career and technical education (CTE) as well as in STEM fields for high school and college students.
	Using findings from various other published studies, the author discussed the progression of the Carl D. Perkins Act of 1984, which required states to employ a sex equity coordinator to mitigate gender bias and stereotypes, over the past three decades, and how it has affected women’s involvement in nontraditional careers, such as electrical work and plumbing. The study also briefly summarized other gender equity trends in CTE and STEM fields, including why women choose to not work in CTE and STEM fields and how gender disparities in various industries can be detrimental to society at large.
	The author found that the Perkins Act and its various reauthorizations had aimed to improve female participation in nontraditional career fields, but lack of funds limited the legislation’s influence. The author also identified additional factors influencing women’s career choices—including gender-role socialization, socioeconomic status, parents’ education level and expectations for their daughters, and advice from guidance counselors—as perpetuating gender disparities in these fields. The author concluded that gender disparities in STEM and CTE careers can lead to an underdevelopment of human resources and capital at the societal level.
	The study concluded with a list of suggested strategies to counter existing gender inequities in CTE and STEM, including offering mentoring programs to provide female students with role models in CTE fields, removing gender-biased images or messaging from textbooks and other educational materials, increasing parental education on nontraditional career options for women, and training guidance counselors to address gender equity issues.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Do stronger age discrimination laws make Social Security reforms more effective?","Neumark, D., & Song, J. (2013). Do stronger age discrimination laws make Social Security reforms more effective? Journal of Public Economics, 108, 1-16.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Mixed impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Mod/high-Favorable impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined whether the strength of state age discrimination protections affected rates of Social Security benefit claiming and employment outcomes of older workers.
The study used a nonexperimental design and the data on Social Security claiming and employment outcomes from the Health and Retirement Study (HRS) to estimate the impacts.
The study found that among older men affected by the 1983 Social Security reform, age discrimination protections were associated with increases in employment rates between age 62 and the full retirement age and delayed benefit claiming.
The quality of casual evidence presented in this report is moderate because the study was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to state age discrimination laws, but other factors might also have contributed.","The 1983 Social Security Reform","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2013,https://ideas.repec.org/a/eee/pubeco/v108y2013icp1-16.html,"Employment and Training Review Protocol"
"Exploring women engineering faculty’s mentoring networks","Long, Z., Buzzanell, P., Kokini, K., Wilson, R., Batra, J., & Anderson, L. (2013). Exploring women engineering faculty’s mentoring networks. Proceedings of the 2013 American Society for Engineering Education Annual Conference & Exposition, Washington, DC.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study described how women in academic STEM positions (compared with corporate environments) constructed their own mentoring networks; it also explored the mentoring configurations female academics preferred as support for their careers.
		The authors conducted in-depth semistructured interviews with seven female engineering faculty at a single large Midwestern university, creating detailed maps of the faculty members’ mentoring networks and evaluating the nature, importance, and perceived value of each connection.
		The study found that the female faculty were generally dissatisfied with formal mentoring relationships and instead sought mentors on their own. Faculty selected these informal mentors from among those with whom they had an existing relationship, such as a friend or co-committee member. Mentees selected multiple mentors with mixed backgrounds and characteristics to serve different purposes (for example, discussing work-life balance or career advancement) and mentees typically had at least one mentor outside of the department or university to discuss more confidential matters. Findings were generally consistent with mentor relationships in a corporate setting.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2013,http://www.asee.org/public/conferences/20/papers/6463/download,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Can post-employment services combined with financial incentives improve employment retention for welfare recipients?","Dorsett, R., Hendra, R., Robins, P. K., & Williams, S. (2013). Can post-employment services combined with financial incentives improve employment retention for welfare recipients? Evidence from the Texas Employment Retention and Advancement evaluation. (No. 1413-13). Madison, WI: Institute for Research on Poverty.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of the Texas Employment Retention and Advancement (ERA) program on employment entry and retention in two sites: Corpus Christi and Fort Worth.
The authors conducted a non-experimental study using data originally collected for a randomized controlled trial. They analyzed survey data and state Unemployment Insurance records to estimate the effects of the ERA program on employment entry and retention.
The study found a positive and statistically significant relationship between participation in the ERA program and the likelihood of transitioning into employment and between ERA and employment retention at the Corpus Christi site.
The quality of causal evidence presented in this report is low because the authors did not account for differences in wages or employment between the ERA and comparison groups before the beginning of the ERA program. This means we are not confident that the estimated effects are attributable to the Texas ERA program; other factors are likely to have contributed.","Texas Employment Retention and Advancement (ERA)","Job search assistance and supportive services Other wages and benefits","Unemployed, Low income, Female",,"United States",2013,http://www.niesr.ac.uk/sites/default/files/publications/dp409_0.pdf,"Review Protocol"
"An outcome evaluation of an adult education and postsecondary alignment program: The Accelerate New Mexico experience","Rivera, M., Davis, M., Feldman, A., & Rachkowski, C. (2013). An outcome evaluation of an adult education and postsecondary alignment program: The Accelerate New Mexico experience. Problems and Perspectives in Management, 11(4), 105-120.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:

The study’s objective was to examine the impact of the Accelerate New Mexico Math Camp on outcomes for community college students, including women and minority students traditionally underrepresented in STEM.
The study used course-administered assessments to compare the math competency and attitudes about math of students before and after participation in the Accelerate summer program.
The authors found that students (both male and female) who participated in the Accelerate program had higher math competency and less anxiety about math after completing the program, compared with their competency and anxiety measured before enrollment in the program.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the Accelerate New Mexico Math Camp program; other factors are likely to have contributed.","the Accelerate New Mexico Math Camp","Science, Technology, Engineering, and Math (STEM) programs Community college education and other classroom training Youth programs","Youth, Other barriers, Female",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns","Smith, J., Lewis, K., Hawthorne, L., & Hodges, S. (2013). When trying hard isn’t natural: Women’s belonging with and motivation for male-dominated STEM fields as a function of effort expenditure concerns. Personality & Social Psychology Bulletin, 39(2), 131-143. [Study 2]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Unfavorable impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to examine the impact of male-dominated marketing of a fictitious eco-psychology program on female students’ interest in pursuing the program.
Authors randomly assigned 49 female students in an introductory psychology course to receive a brochure about a fictitious one-year master’s program in eco-psychology that was presented as either male-dominated or gender-neutral. Students were then asked to rate their motivation to enroll in the program.
The study found that female students who read the male-dominated brochure displayed statistically significantly lower interest in pursuing the program than those in the gender-neutral group.
The quality of causal evidence presented in this report is low because the authors did not include sufficient controls in their analysis to account for differences between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to type of brochure reviewed; other factors are likely to have contributed.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The role of parents and teachers in the development of gender-related math attitudes","Gunderson, E., Ramirez, G., Levine, S., & Beilock, S. (2012). The role of parents and teachers in the development of gender-related math attitudes. Sex Roles, 66, 153-166.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study’s objective was to summarize existing literature and identify future areas for research related to the relationship between parents’ and teachers’ gender stereotypes in math and children’s mathematics performance and self-concept.
	The authors summarized the findings of recent studies that examined parents’, teachers’, and students’ math gender stereotypes, focusing on impressions of success—whether boys or girls had above-average ability in math or were likely to succeed in math—as well as reasons for that success, particularly natural ability and effort.
	The authors suggested three new directions for future research on this topic based on their review of the literature. First, research could investigate the links among parents’ and teachers’ math anxiety, self-efficacy, and teaching self-efficacy and their children’s or students’ attitudes. A second direction involved transmission of gender differences in theories of math intelligence—that is, whether mathematical ability is fixed or can improve through effort. Finally, research has shown that children identify with math gender stereotypes from a young age, but does not yet suggest a mechanism through which children acquire these stereotypes.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The more things change, the more they stay the same? Prior achievement fails to explain gender inequality in entry into STEM college majors over time","Riegle-Crumb, C., King, B., Grodsky, E., & Muller, C. (2012). The more things change, the more they stay the same? Prior achievement fails to explain gender inequality in entry into STEM college majors over time. American Educational Research Journal, 49(6), 1048-1073.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study explored whether U.S. students’ prior achievement in math and science contributed to higher enrollment rates for men versus women in physical science and engineering degree programs in the 1980s, 1990s, and 2000s.
	The authors analyzed data from the High School and Beyond Study (1980s), the National Education Longitudinal Study (1990s), and the Educational Longitudinal Study (2000s) to determine whether gender was correlated with college major choice, after controlling for three different measures of prior academic achievement.
	The study found that female students were approximately 20 percentage points less likely to declare a science, technology, engineering, and mathematics (STEM) major than male students. However, prior achievement in math and science, as measured by GPA, average standardized test scores, and position along the test score distribution, did very little to explain this gender gap.
	The study found that girls’ higher performance in English versus math and science only somewhat accounted for the gender gap in college major choice. Factors such as ethnicity and socioeconomic status were more influential.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Exploring bias in math teachers’ perceptions of students’ ability by gender and race/ethnicity","Riegle-Crumb, C., & Humphries, M. (2012). Exploring bias in math teachers’ perceptions of students’ ability by gender and race/ethnicity. Gender & Society, 26(2), 290-322.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This study aimed to inform the literature on gender stereotypes in classrooms by studying whether teachers displayed bias by underestimating the ability of 10th-grade female and minority students in high school math classes, even after taking into account students’ grades and test scores.
	The study analyzed survey data from students and teachers across the United States who were part of the Educational Longitudinal Study of 2002, which followed 15,000 10th-grade students through college and into the labor force. The authors estimated multinomial logistic regression models to determine whether teachers were more likely to underestimate the academic ability of female, black, and Hispanic students compared to white male students, controlling for grade point average (GPA), test scores, and student, teacher, and school characteristics. Results were generated separately for students in low-level, average-level, and advanced-level high school math courses.
	The study found that in low-level and advanced-level math classes, teachers were less likely to say that the course was too easy for white female students as compared to white male students. In average-level math classes, teachers were both less likely to say that the course was too easy and more likely to say that it was too difficult for white female students, compared to white male students. These findings did not extend to minority female students nor to minority male students, who, after controlling for GPA and test scores, were just as likely to be rated by teachers as being in a course that was too easy or too difficult for them as white male students.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Expanding Your Horizons: The impact of a one-day STEM conference on middle school girls’ and parents’ attitude toward STEM careers","Massi, L., Reilly, C., Johnson, D., & Castner, L. (2012). Expanding Your Horizons: The impact of a one-day STEM conference on middle school girls’ and parents’ attitude toward STEM careers. Proceedings of the 2012 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to estimate the effects of a one-day annual conference on middle school girls’ (grades 6–8) and their parents’ interest in and knowledge of science, technology, engineering, and mathematics (STEM). The goal of the conference, which was held at a large metropolitan, public university in central Florida, was to encourage young girls to consider STEM careers.
The authors administered a single questionnaire to conference attendees that asked about attendees’ knowledge of and interest in STEM before and after the conference. The survey was included in backpacks distributed at the beginning of the day, and participants were reminded to complete it at the end of the day. Of the 214 middle school girls and 103 parent attendees, 80 percent of the girls and 37 percent of the parents turned in a completed survey.
The authors did not conduct any statistical tests to measure changes in knowledge of and interest in STEM before and after participating in the conference.
The quality of causal evidence presented in this report is low because the authors measured pre-intervention knowledge of and interest in STEM after participation in the intervention, which may be subject to recall bias. This means we are not confident that any estimated effects would be attributable to the Expanding Your Horizons (EYH) conference, but other factors are likely to have contributed. Further, the authors did not conduct statistical analysis to measure whether observed changes in outcomes were actually significant.","the Expanding Your Horizons (EYH) Conference","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2012,https://www.asee.org/public/conferences/8/papers/3204/download,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Income tax and older American workers’ job transition into self-employment.","Choi, S. (2012). Income tax and older American workers’ job transition into self-employment. Korea and the World Economy, 13(2), 189-214.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the effect of a progressive income tax on the likelihood of transitioning to self-employment among older workers.
The study used a nonexperimental approach (the exponential hazard model) that compared the likelihood of transitioning from a wage job to self-employment based on the marginal taxes a worker would expect to pay. The author used the data from the Health and Retirement Study in 1992 matched to the Social Security Administration Master Earnings File from 1980 to 1991, which contains W-2 records of earnings from both wage work and self-employment.
The study found that a more progressive tax (that is, a tax schedule that increases sharply toward the higher end) or higher marginal tax rates reduces the likelihood of entering self-employment.
The quality of causal evidence presented in this report is low because the author did not ensure that the groups with higher or lower marginal tax rates were similar before a change in the tax schedule. This means we are not confident that the estimated effects are attributable to changes in marginal taxes; other factors are likely to have contributed.","Progressive Income Tax","Other employment and reemployment","Adult, Older worker, Employed",,"United States",2012,,"Employment and Training Review Protocol"
"Inspiring girls and their female after school educators to pursue computer science and other STEM careers","Koch, M., & Gorges, T. (2012). Inspiring girls and their female after school educators to pursue computer science and other STEM careers. International Journal of Gender, Science and Technology, 4(3), 294-312.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to examine the impact of facilitating Build IT, an extracurricular information technology (IT) program for middle-school girls, on the facilitators’ interest in STEM education and careers. The authors also described outcomes for girls who participated in the facilitator-led program.
The study relied on a survey administered to 31 program facilitators after they had been involved with the program for one year.
The study found no statistically significant relationships between facilitating Build IT and interest in STEM education and careers.
The quality of causal evidence presented in this report is low because the authors did not use a research design that supported causal inference or control for factors other than Build IT that could contribute to the observed outcomes.","Build IT","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Other barriers, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The efficacy of supported employment for middle-aged and older people with schizophrenia.","Twamley, E. W., Vella, L., Burton, C. Z., Becker, D. R., Bell, M. D., & Jeste, D. V. (2012). The efficacy of supported employment for middle-aged and older people with schizophrenia. Schizophrenia Research, 135(1-3), 100-104.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of Individual Placement and Support (IPS) on competitive employment for older people with schizophrenia
The study was a randomized controlled trial (RCT) conducted at a community mental health clinic and used weekly work logs cross-referenced with pay stubs to measure employment outcomes over a 12-month period.
The study found that IPS was positively associated with employment and earnings.
The quality of causal evidence presented in this report is low because the RCT has a confounding factor. This means we are not confident that the estimated effects are attributable to the IPS program; other factors are likely to have contributed.","Individual Placement and Support and Vocational Rehabilitation Program","Health Individuals facing barriers to employment Job search assistance and supportive services","Adult, Older worker, Disability",,"United States",2012,https://psycnet.apa.org/record/2011-30184-001,"Employment and Training Review Protocol"
"Transformative graduate education programs: An analysis of impact on STEM and non-STEM Ph.D. completion","Kniola, D., Chang, M., & Olsen, D. (2012). Transformative graduate education programs: An analysis of impact on STEM and non-STEM Ph.D. completion. Higher Education, 63(4), 473–495. doi:10.1007/s10734-011-9453-8","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to determine whether universities participating in Transformative Graduate Education Programs (TGPs) conferred a greater number of science, technology, engineering, and mathematics (STEM) doctoral degrees, particularly to women and underrepresented candidates of color. The study also examined whether TGPs resulted in a greater number of non-STEM doctoral degrees, but that is not the focus of this review.
The authors analyzed data from the Integrated Postsecondary Education Data System to estimate the impact of a university’s participation in one of seven TGPs during the 2003–2005 period, controlling for institutional characteristics.
The study found that TGPs were associated with a higher number of doctoral degrees awarded in STEM fields to women across all institutions.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to universities’ participation in TGPs. Other factors are likely to have contributed.","Transformative Graduate Education Programs (TGPs)","Science, Technology, Engineering, and Math (STEM) programs","Other barriers, Female, STEM professional",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Gender and career outcomes of U.S. engineers","Frehill, L. (2012). Gender and career outcomes of U.S. engineers. International Journal of Gender, Science and Technology, 4(2), 149-166.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This study addressed two research questions concerning the retention rates of engineers in the United States: whether more women leave the engineering field because of family-related concerns than men and whether more women than men move into managerial work.
	The author analyzed data from the National Science Foundation’s public use Science and Engineering Statistical Data System (SESTAT) for 2006. SESTAT includes three surveys: the National Survey of College Graduates, the National Survey of Recent College Graduates, and the Survey of Doctorate Recipients. The study sample was 17,004 people.
	The study found that the retention rate for women in the engineering field was 70 percent, compared with 86 percent retention for men, and more women cited family-related issues as a reason for departure than did men. However, family-related issues were not the main reason for leaving the engineering field for either gender—female respondents noted a change in career or professional interests as their main reason for departure, whereas a lack of pay or promotion opportunities were the main reasons cited by men.
	The study also found that women engineers were less likely to currently hold management positions compared with male respondents. For example, 49 percent of male respondents indicated that they supervised employees in their current position, compared with 33 percent of female respondents.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, Male, STEM professional",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Success of joint programs between junior and senior colleges","Krudysz, M., Walser, A., & Alting, A. (2012). Success of joint programs between junior and senior colleges. American Society for Engineering Education.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:


The study’s objective was to examine the impact of the Joint/Dual (JD) program on retention and graduation rates for community college and undergraduate engineering students who were under-represented minorities and women. JD students received dual admission to a community college as well as a senior college, and transitioned into the upper division of the baccalaureate engineering program at the senior college upon successful completion of lower-division courses and degree requirements at the community college.
This study used data from junior and senior colleges within a major urban university system to compare outcomes of a treatment group of JD program participants and a comparison group of transfers into the urban university system from other, nonparticipating community colleges.
This study found that first- and second-year retention rates were higher for JD program participants than for transfers from other community colleges, but did not conduct statistical tests of these differences.
The quality of causal evidence presented in this report is low because the authors did not include controls for relevant student characteristics in the analysis. This means we are not confident that the estimated effects are attributable to the JD program. Other factors are likely to have contributed.","the Joint/Dual Program","Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Female, Other",,"Urban, United States",2012,https://www.asee.org/public/conferences/8/papers/3558/download,"Community College Review Protocol"
"Women in industrial engineering: Stereotypes, persistence, and perspectives","Brawner, C., & Camacho, M. (2012). Women in industrial engineering: Stereotypes, persistence, and perspectives. Journal of Engineering Education, 101(2), 288-318.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This study explored why more women major in industrial engineering in college than other engineering fields.
	The authors tabulated quantitative data from eight colleges and universities in the southeastern region of the United States using data from the Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). They also conducted focus groups with 20 female industrial engineering students and qualitatively analyzed the content of industrial engineering department websites at the eight MIDFIELD institutions.
	The study found that students transferred into an industrial engineering major program after taking several semesters of coursework in another field at the university. It found no evidence that women with weak academic records were more likely to choose industrial engineering over other engineering majors.
	Focus group participants felt that their department had a warm, familial atmosphere, and that the major provided a more general background in engineering, which could offer them more flexible job opportunities in a variety of related fields. This was consistent with departmental websites, which marketed themselves as having a strong sense of community and collegiality, and stressed the wide range of career opportunities available to their graduates as problem solvers for society and industry.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Women, men, and academic performance in science and engineering: The gender difference in undergraduate grade point averages","Sonnert, G., & Fox, M. (2012). Women, men, and academic performance in science and engineering: The gender difference in undergraduate grade point averages. Journal of Higher Education, 83(1), 73-101.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study tested a hypothesis that women have higher undergraduate grade point averages (GPAs) than their male peers (referred to as a GPA advantage), and that the difference is most pronounced in STEM fields with fewer female students and fewer female faculty members. In addition, the authors tested a second hypothesis that women’s GPA advantage was driven by institution type, with larger advantages existing at institutions offering specific programs supporting women in science and engineering, compared with institutions without such programs.
	The authors used data from the Integrated Postsecondary Data System and from a survey of 499 university and college registrars in the United States to examine the academic performance of undergraduate students in biology, the physical sciences, and engineering from 1984 to 2000. They used regression models to compare the GPAs of male and female students over time both across and within the three fields, controlling for the percentage of female majors, degree recipients, and faculty within each field at each institution in a given year.
	The study found that women’s GPAs across the three fields were, on average, 0.1 points (on a 4.0-point scale) higher than men’s during the study period. The study also found that in the physical sciences field, the percentage of female degree recipients was significantly associated with women’s GPA advantage—when more women earned degrees in physical sciences, the GPA advantaged diminished; however, this finding did not hold in the biology or engineering fields.
	The study found that the strongest predictor of women’s GPA advantage across fields was the type of institution, with a larger advantage occurring at nonresearch institutions compared with research institutions. In addition, the study found that at nonresearch institutions with programs specifically supporting women in science and engineering, women’s GPA advantage was even larger.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The effects of hands-on activities on middle school females’ spatial skills and interest in engineering and technology-based careers","Phelps, M. (2012). The effects of hands-on activities on middle school females’ spatial skills and interest in engineering and technology-based careers. Proceedings of the 2012 American Society for Engineering Education Annual Conference & Exposition. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to examine the impact of participating in technology-based three-dimensional hands-on activities on middle school girls’ interest in becoming an engineer or technologist. The activities took place at a 2006 Society of Women Engineers-sponsored event and included designing and constructing a vehicle and assembling an electrical circuit.
The author conducted statistical analyses to compare self-reported career interests of girls who participated in the hands-on activities and those who did not, as measured by a program-administered survey.
The study found that participation in the hands-on activities was positively related to increased interest in becoming an engineer or technologist.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the hands-on activities; other factors are likely to have contributed.","the Intervention","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female","Professional, scientific, and technical services","United States",2012,https://peer.asee.org/the-effects-of-hands-on-activities-on-middle-school-femal…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Tax elasticity of labor earnings for older individuals","Alpert, A. & Powell, D. (2012). Tax elasticity of labor earnings for older individuals. (Report no. 2012-272). Ann Arbor, MI: University of Michigan Retirement Research Center, University of Michigan.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of tax changes on people’s employment, retirement, and labor income outcomes.
The study uses a nonexperimental approach and the data from the Health and Retirement Study to estimate the effect.
The study found positive statistically significant relationships between increases in after-tax labor income and the employment of men and women as well as the delay in retirement for men. The study also found a positive statistically significant relationship between increases in the marginal net-of-tax rate and earnings for women.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to tax changes from the Economic Growth and Tax Relief Reconciliation Act (EGTRRA), the Jobs and Growth Tax Relief Reconciliation Act (JGTRRA), and their marriage tax penalty relief provision, but other factors might also have contributed.","Tax Changes","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2012,https://deepblue.lib.umich.edu/bitstream/handle/2027.42/94533/wp272.pdf?sequenc…,"Employment and Training Review Protocol"
"Adolescent girls’ experiences and gender-related beliefs in relation to their motivation in math/science and English","Leaper, C., Farkas, T., & Brown, C. (2012). Adolescent girls’ experiences and gender-related beliefs in relation to their motivation in math/science and English. Journal of Youth and Adolescence, 41(3), 268-282.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study examined factors that influence teenage girls’ academic motivation in math, science, and English.
	The authors analyzed survey data from 579 girls ages 13 to 18 years old in Georgia and California to estimate predictors of the girls’ academic motivation in math, science, and English. The survey asked about the youths’ (1) academic motivation for each subject; (2) grades in these subjects; (3) perceptions of support from family and peers in these subjects; (4) attitudes on gender identity, measured by perceived parental and peer pressure to conform with traditional gender roles, the degree to which the youth perceived themselves to be so-called typical girls, and youths’ level of contentedness with the gender expectations they face; (5) opinions on gender egalitarianism of specific tasks, such as making decisions for the family; and (6) exposure to feminism, as measured by whether the youth had heard of the women’s rights movement and feminism through media or conversations with family or friends, as well as whether the youth knew that their mothers, teachers, or classmates identified as feminists.
	The study found that the following factors were positively associated with the girls’ motivation in math and science: mother and peer support in these subjects, belief in gender equality, less pressure from parents to conform to gender norms, holding gender-egalitarian beliefs, and exposure to feminism ideas and/or feminists.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Increasing opportunities for low-income women and student parents in science, technology, engineering, and math at community colleges","Costello, C. (2012). Increasing opportunities for low-income women and student parents in science, technology, engineering, and math at community colleges. Washington, DC: Institute for Women’s Policy Research.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		This study examined the enrollment patterns for low-income women and students with children (including mostly mothers and some fathers, collectively referred to as student parents) in STEM fields at community colleges in the United States, comparing outcomes for 2000–2001 to 2008–2009. In addition, the study presented examples of promising strategies to encourage low-income women and student parents to participate in STEM education at community colleges.
		To assess the involvement of low-income women and student parents in STEM fields at community colleges, the author analyzed publicly available data, including data from the U.S. Department of Labor and the National Center for Education Statistics at the U.S. Department of Education. The author also completed a review of existing programs at community colleges that encourage low-income women and student parents to participate in STEM fields, and consulted with 16 experts to formulate suggestions for additional programs to support these students in STEM fields.
		The study found that in the 2008–2009 school year, women earned 22 percent of associate’s degrees awarded in STEM fields—a 7 percentage point reduction in the proportion of STEM associate degrees earned by women eight years earlier. The study also found that, in 2008, roughly one-third of community college students were parents and student parents with children younger than 12 who ultimately earned a baccalaureate or master’s degree in a STEM field were more likely to have attended a community college at some point than students without young children.
		Given the decline in women’s share of STEM associate degrees earned over the study observation period, as well as the high proportion of student parents in STEM fields who attend community colleges (compared with students without young children), the author highlighted potential strategies to increase and improve the involvement of low-income women and student parents in STEM fields at community colleges, including targeted recruitment, financial support, child care services, strong developmental education options, and specific counseling and academic support for women. These recommendations were informed by information gathered from existing programs and consultation with experts.",,"Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Female, Male, Parent, Low income",,"United States",2012,http://www.iwpr.org/initiatives/student-parent-success-initiative/increasing-op…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Engineering students' beliefs about research: Sex differences, personality, and career plans","Woodcock, A., Graziano, W., Branch, S., Ngambeki, I., & Evangelou, D. (2012). Engineering students' beliefs about research: Sex differences, personality, and career plans. Journal of Engineering Education, 101 (3), 495-511.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The authors explored the relationships among male and female engineering major undergraduates’ perceptions of research and researchers, their intellectual orientation toward people versus objects, and their interest in pursuing engineering research careers.
	The authors administered an online study to second-year or higher undergraduate students in the engineering, life sciences, and psychology departments of a large research university in the midwestern United States. They used regression analysis to identify beliefs and personality traits that predict interest in engineering research careers for male and female students.
	The study found that having a “Thing Orientation”—a predisposition to focus on objects and their manipulation instead of a tendency to notice and identify with others’ emotions—significantly predicts research interest and has a greater association with research interest among female engineering students than among their male classmates.
	The authors also sought to discover the pathway through which Thing Orientation promotes interest in a research career by identifying associations between Thing Orientation and beliefs about research. For the female students in this sample, Thing Orientation was positively associated with the beliefs that research is innovative and transformative, requires repetitive lab work, and calls for creativity and collaboration. In turn, these beliefs affect female engineering students’ interest in research careers; the belief that research is innovative and transformative is positively associated with interest in a research career.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2012,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Living-learning programs for women in STEM","Inkelas, K. (2011). Living-learning programs for women in STEM. New Directions for Institutional Research 152, 27-37.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study’s objective was to review the evidence of the effects of living-learning programs (LLPs)—shared residences at postsecondary institutions designed to bridge in- and out-of-class learning and establish tight-knit communities—on academic and other outcomes for women in STEM disciplines.
		The author reviewed studies that compared the short- and long-term academic outcomes of female STEM majors who participated in LLPs, particularly women-only STEM LLPs, to male and female STEM majors who did not participate in such programs. Many of these studies relied on the 2007 National Study of Living-Learning Programs, which encompassed programs at more than 50 U.S. colleges and universities and represented the responses of 22,519 students; other university-specific studies from Michigan State University, the University of Wisconsin-Madison, and the University of Michigan were included.
		The study found that participants in a women-only STEM LLPs were more likely to attend graduate school in a STEM field than were female STEM majors who participated in coeducational STEM LLPs, non-STEM LLPs, or no LLP, by 35, 31, and 29 percent, respectively.
		Participants in a women-only STEM LLP at the University of Michigan were more likely to persist in STEM majors than were male or female nonparticipants with similar academic interests and achievement.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Mothers of invention: Gender, motherhood, and new dimensions of productivity in the science profession","Whittington, K. (2011). Mothers of invention: Gender, motherhood, and new dimensions of productivity in the science profession. Work and Occupations, 38(3), 417-456.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to determine whether women, specifically mothers, in STEM academic or industry jobs were less likely to participate in the patenting process than fathers and childless men in the same field. In addition, the study sought to examine whether previous experience with patenting reduced gender or motherhood inequalities in patenting participation.
		The study analyzed survey data from the April 1995 and April 2001 waves of the Survey of Doctorate Recipients, a longitudinal study of research doctorates conducted by the National Science Foundation. The author restricted the sample to scientists in four-year colleges and in business or industry who worked in computer and mathematical sciences, life sciences, physical sciences, and engineering; were active in applied and basic research, development, or design; and worked full time. Respondents were asked if they had been named as an inventor on a U.S. patent application in the past five or six years. The author estimated logistic regression models to determine whether females, and specifically mothers, were less likely to have received a patent in the past five or six years. The author also conducted additional analyses to determine if previous experience with patenting affected patenting participation.
		The study found that in academia, among those surveyed at both rounds, mothers were no less likely to patent than males, including fathers, or childless females after controlling for prior patenting experience. In industry, among those surveyed in both rounds, married, childless women were less likely to patent than childless women, even after controlling for prior patenting experience. The authors estimated numerous other models that found mixed effects of gender, marital status, and parenthood on the likelihood of scientists participating in patenting activities.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, STEM professional, Parent",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Middle school girls’ perceptions of engineers before and after a female only summer enrichment program","Hirsch, L., Berliner-Heyman, S., Cano, R., Kimmel, H., & Carpinelli, J. (2011). Middle school girls’ perceptions of engineers before and after a female only summer enrichment program. Paper presented at the 2011 IEEE Frontiers in Education Conference.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

		In a study performed over the course of a summer engineering program for 4th- through 8th-grade girls, the authors sought to determine whether participants’ attitudes toward engineering changed from the beginning to end of the program and whether traditional surveys fully captured participants’ growth over the course of such an experience.
		The authors administered two tests, the Middle School Attitude to Mathematics, Science and Engineering Survey (MATE) and the Draw an Engineer Test (DAET), to about 100 participants in a summer 2009 New Jersey Institute of Technology engineering program for 4th- to 8th-grade girls. The MATE comprises seven psychological subscales focused on students’ interest in various aspects of engineering, attitudes toward mathematics and science, problem-solving, knowledge of engineering, and perceptions of males’ and females’ relative competence in mathematics and science; the DAET prompts respondents to draw an engineer at work and summarize their drawing in a sentence. Participants took both tests before beginning and after completing the program.
		The study found that girls’ knowledge of engineering increased significantly; at the beginning of the study, fewer than half of the participants could name one or more kinds of engineers, but by the end of the program more than 85 percent of participants correctly identified at least one kind of engineer.
		Differences in pre- and post-program engineer drawings and descriptions suggested more nuanced changes in girls’ understanding of engineering. For example, the proportion of responses depicting engineers communicating with others more than doubled, from only 12 percent before the program to 30 percent afterward. The authors concluded that the DAET complements the MATE and warranted further study as a measure of children’s changing perceptions of science and scientists.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,http://fie-conference.org/fie2011/papers/1471.pdf,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Arguing separate but equal: A study of argumentation in public single-sex science classes in the United States","Glasser, H. (2011). Arguing separate but equal: A study of argumentation in public single-sex science classes in the United States. International Journal of Gender, Science and Technology, 3(1), 70-92.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	In a study performed at a public coeducational middle school during the 2007–2008 school year, the author sought to determine whether students in single-sex science classes learned “the same science” by experiencing the same instructional approaches and earning the same grades as those in mixed-gender classes.
	The author attended both the boys’ and girls’ science classes, and collected data from audio and video recordings, reviews of students’ work, and interviews with students and teachers. He analyzed these data using a variety of coding schemes, with particular emphasis on argumentation, defined for the purpose of the study as a whole-class scientific discussion involving the teacher that contained both a challenge and a defense. He also examined more than seven hours of video recordings from each class section and counted the instances of argumentation occurring in each class. Finally, the author reviewed students’ grades in the course.
	Although argumentation was not a primary feature of either boys’ or girls’ science classes, with no more than five instances of argumentation per hour of instruction in either case, the author found that boys’ science classes contained more than three times as many instances of argumentation than did girls’ classes. Over the sample lessons considered in this study, the boys produced 22 instances of argumentation, compared with only 7 in the girls’ class. There were no differences in students’ grades by gender.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"WomenLEAD: Leadership development for female faculty in business and engineering","Hodges, J., Pearson, A., & Reese, D. (2011). WomenLEAD: Leadership development for female faculty in business and engineering. International Journal of Gender, Science and Technology, 3(2), 331-337.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study’s main objective was to describe and promote the implementation of WomenLEAD, a program offered at a business and engineering college to foster leadership development in female faculty by providing advice and support.
	WomenLEAD consisted of eight sessions focused on challenges faced by female faculty for advancement, such as work-life balance, gender biases, and communication differences. At the end of the program, the authors, who created and implemented the WomenLEAD program, administered a participant survey to elicit participants’ feedback on the degree to which each of the sessions influenced participants’ awareness and understanding of issues relevant to their leadership development.
	The participants’ survey suggested that WomenLEAD increased women’s awareness and understanding of issues covered in each session, such as communicating with power, career challenges, and leadership qualities. Comments on the survey indicated appreciation for the support network of women the program created. The program intends to continue this networking through continued formal and informal gatherings and to expand it to female faculty at other colleges.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Stemming inequality? Employment and pay of female and minority scientists and engineers in the federal and private sectors","Oh, S., & Lewis, G. (2011). Stemming inequality? Employment and pay of female and minority scientists and engineers in the federal and private sectors. Social Science Journal, 48(2), 397-403.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study investigated the relationships among sector (federal or private); science, technology, engineering, and mathematics (STEM) occupation; gender or racial minority status; and earnings to determine whether federal- or private-sector STEM positions offer the greatest financial opportunities for female and racial minority STEM professionals.
	The authors compared the earnings of male and female STEM and non-STEM public sector employees in 1983 and 2003 using a 1 percent sample of the Office of Personnel Management’s Central Personnel Data File. To compare public- and private-sector outcomes, the authors also analyzed a 5 percent sample of the 2000 U.S. Census, examining differences in STEM and non-STEM public and private employees’ salaries by gender and race.
	The study found that women in public sector STEM positions earned 7.2 percent less than men in comparable public sector positions in 1983 even after controlling for education, STEM field, tenure, age, and race. This gender pay gap disappeared in the 2003 sample.
	Race and gender pay disparities, measured as the average difference in pay between racial minorities or women and white men, are smaller in the federal than the private sector and for STEM relative to non-STEM occupations. For example, in 1999, black male federal STEM employees earned 8.3 percent less than white men, compared to a pay gap of 10.9 percent for black male private sector STEM employees. For black men in non-STEM occupations, the pay gap relative to white men was 11.1 percent in the public sector and 20.6 percent in the private sector.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Other barriers, Female, STEM professional","Professional, scientific, and technical services","United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"STEM development: A study of 6th-12th grade girls’ interest and confidence in mathematics and science (Doctoral dissertation, Iowa State University)","Heaverlo, C. (2011). STEM development: A study of 6th-12th grade girls’ interest and confidence in mathematics and science (Doctoral dissertation, Iowa State University). Retrieved from http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1035&context=etd","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study’s objective was to determine whether middle and high school girls’ levels of interest and confidence in science and mathematics differed and to identify factors that promote their interest and confidence in these subjects.
		The author administered a survey to the 6th–12th grade attendees of the three April 2009 sessions of Iowa State University’s Taking the Road Less Traveled Career Conference. The author performed descriptive statistical analyses.
		The study found that middle and high school girls’ interest and confidence in science and mathematics did not differ.
		Teacher influence was a statistically significant predictor of all four outcomes examined—math interest, science interest, math confidence, and science confidence. In particular, math teacher influence predicted math interest and confidence, and science teacher influence predicted science interest and confidence. The availability of extracurricular science, technology, engineering, and mathematics activities also predicted math interest and confidence.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1035&context=etd,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Labor market rigidities and the employment behavior of older workers.","Blau, D., & Shvydko, T. (2011). Labor market rigidities and the employment behavior of older workers. Industrial and Labor Relations Review, 64(3), 464-484.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of flexibility in hours of work on the likelihood that older workers separate from an employer.
The study used a statistical model to compare the job exit behavior of older and younger workers in firms with different shares of women younger than age 30. The analysis used monthly data on workers ages 45 to 69 in the Survey of Income and Program Participation from 1990 to 2001 matched to employers in the Longitudinal Employment and Household Dynamics data.
The study found that workers in their sixties who work for employers with a higher share of women younger than age 30 are less likely to experience job separations compared with workers in their late 40s to mid-50s.
The quality of causal evidence presented in this report is low because the groups being compared might not be similar before leaving their employer. This means we are not confident that the estimated effects are attributable to employment flexibility, other factors are likely to have contributed.","Employment flexibility","Other employer services","Adult, Older worker",,"United States",2011,https://www.jstor.org/stable/41149475?seq=1#page_scan_tab_contents,"Employment and Training Review Protocol"
"Questioning a white male advantage in STEM: Examining disparities in college major by gender and race/ethnicity","Riegle-Crumb, C., & King, B. (2011). Questioning a white male advantage in STEM: Examining disparities in college major by gender and race/ethnicity. Educational Researcher, 39(9), 656-664.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to examine differences in students’ likelihood of majoring in a science, technology, engineering, and mathematics (STEM) field, based on gender and ethnicity.
		The study used statistical methods to estimate the likelihood of majoring in the physical sciences or engineering, or in the biological sciences, versus non-STEM fields. The authors also examined this likelihood for traditionally underrepresented groups (white, black, and Hispanic women and black and Hispanic men) compared to white men. The analyses controlled for student socioeconomic factors, delayed college entry, academic preparation, and attitudes toward math. Data sources included students’ high school transcripts and surveys conducted through the Educational Longitudinal Study of 2002.
		The study found that white and Hispanic women were less likely to major in the physical sciences or engineering, versus a non-STEM major, than white men. Black women were also less likely to major in a physical science or engineering field than white men, but the gap was slightly smaller.
		In contrast to the large gender gaps evident in the choice of physical science and engineering majors, there were no differences between the gender/ethnicity groups in the likelihood of majoring in a biological sciences field compared to a non-STEM field. The study found that academic preparation and attitudes toward math rarely explained why men were more likely to choose a STEM major than women.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, Male, White, Black or African American, Hispanic of any race",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Do female and male role models who embody STEM stereotypes hinder women’s anticipated success in STEM?","Cheryan, S., Siy, J., Vichayapai, M., Drury, B. & Kim, S. (2011). Do female and male role models who embody STEM stereotypes hinder women’s anticipated success in STEM? Social Psychological and Personality Science, 2(6), 656-664.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:

The study’s objective was to examine the impact of interacting with an upper-level college role model on female students’ beliefs that they could succeed as a computer science major.
The study used a randomized controlled trial to assign female students majoring in a discipline other than computer science to meet with one of four different types of upper-level students who varied by gender and whether they embodied perceived stereotypes of computer science majors. The authors administered a questionnaire to estimate impacts of the different intervention conditions.
The study found that women who interacted with a stereotypical computer science role model believed they would be less successful as a computer science major compared with women who interacted with a nonstereotypical computer science role model and with women who did not interact with a role model.
The quality of causal evidence provided in this study is low because the study had high attrition and the authors did not demonstrate baseline equivalence between the samples or include adequate controls to ensure the resulting groups were similar on all relevant characteristics. This means we are not confident that the effects estimated in this study are attributable to role model interaction; other factors are likely to have contributed.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Understanding the factors affecting degree completion of doctoral women in the science and engineering fields","Ampaw, F., & Jaeger, A. (2011). Understanding the factors affecting degree completion of doctoral women in the science and engineering fields. New Directions for Institutional Research, 152, 59-73.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This study aimed to determine if student characteristics; financial aid receipt; and labor market conditions (such as unemployment, weekly wages, and professors’ salary) predicted doctoral degree completion in science, engineering, or math differently for men and women.
	Following students enrolled in a doctoral program at an academic institution in the southeastern United States, the study used data from the institution in which study participants were enrolled, Bureau of Labor Statistics data on unemployment and weekly earnings, and expected earnings information from the National Faculty Salary Survey by Discipline and Rank in Four Year Colleges and Universities. Nonlinear regression models were used to estimate the effects on doctoral degree completion.
	The study found that 9 percent more males than females in the study completed their doctoral degree, although women tended to be younger at the start of their degree. International students, but not minority students overall, were more likely to complete their degree than white domestic students.
	Regarding financial aid, students who held research assistantships were 67 percent more likely to finish their degree than students without an assistantship, but males were much more likely to acquire an assistantship than females.
	Regarding labor market conditions, students were more likely to complete a degree in fields with high unemployment and higher expected earnings.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs Preventing discrimination","Youth, Female, Male",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"How to recruit women and girls to the science, technology, engineering, and math (STEM) classroom","Milgram, D. (2011). How to recruit women and girls to the science, technology, engineering, and math (STEM) classroom. Technology and Engineering Teacher, 71(3), 4-11.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to suggest successful strategies for increasing the number of women and girls in STEM classes.
		The author used studies of other programs and lessons learned from personal experience to inform a set of suggested strategies for improving the recruitment of women and girls into STEM classes.
		The author suggested that women and girls benefit from seeing female role models in STEM, receiving positive encouraging messages, and being assured that women in STEM can maintain a work/life balance. The author also described specific strategies that schools and programs should implement to improve recruitment of females, including reaching out to counselors and communicating the program’s emphasis on recruiting females, personally encouraging female students, developing outreach materials that display females and that incorporate the color pink, and taking into account how females’ perceptions of and motivations toward STEM differ from those of males.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Gender Equity in Science and Engineering: Advancing Change in Higher Education","Bilimoria, D., & Liang, X. (2011). Gender Equity in Science and Engineering: Advancing Change in Higher Education. New York: Routledge.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to evaluate the effectiveness of the ADVANCE institutional transformation program (ADVANCE-IT) sponsored by the National Science Foundation (NSF) in advancing women’s representation among science, technology, engineering, and mathematics (STEM) faculty in U.S. colleges and universities.
The authors used program implementation and outcome data obtained from 19 participating colleges and universities to examine program initiatives, actions, and outcomes of ADVANCE-IT institutions. The analysis of interest in this review compared the changes in proportions of female STEM faculty in institutions that were implementing ADVANCE-IT programs to national reference groups over the course of the program period, by faculty rank and by type of institution.
The study found that the ADVANCE-IT program was effective in increasing the number of female STEM faculty at all ranks in four-year colleges and research universities.
The quality of causal evidence presented in this study is low. This means we are not confident that the estimated effects are attributable to the ADVANCE-IT program. Other factors are likely to have contributed.","the ADVANCE Program","Other disparities or discrimination in employment and earnings Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Are the predictors of women’s persistence in STEM painting the full picture? A series of comparative case studies","Hughes, R. (2011). Are the predictors of women’s persistence in STEM painting the full picture? A series of comparative case studies. International Journal of Gender, Science and Technology, 3(3), 548-570.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to examine the experiences of several female science and engineering majors within the literature context on predictive factors of persistence in STEM majors and careers. The study focused on the roles of parental support and education level, academic preparation, and perceptions of the STEM culture in influencing the career trajectories of female undergraduates at a United States university in 2009–2010.
		The author conducted two interviews each with five women in their fourth year of college who had chosen to major in a STEM field. The first interview collected the participant’s life history, with a focus on factors and moments leading to the decision to pursue a STEM major. The second interview, conducted several months later, recorded whether the participant’s career plans had changed.
		The study found that the participants’ experiences did not always align with what the literature predicted, and that complex factors contributed to a woman’s decision to stay in or leave a STEM major. Although all the women who stayed in a STEM field described the use of coping strategies for persisting in a male-dominated field, these strategies differed among participants. Those who chose to leave their STEM major did not necessarily do so because of a lack of academic preparation, parental support, or competence, but rather seemed to share a feeling that they did not identify with the peers in their STEM field, who they described as “uptight,” “nerdy,” and lacking a life outside of the classroom.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Removing the barriers to full professor: A mentoring program for associate professors","Buch, K., Huet, Y., Rorrer, A., & Roberson, L. (2011). Removing the barriers to full professor: A mentoring program for associate professors. Change: The Magazine of Higher Learning, 43(6), 38-45.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The objective of the study was to develop and assess a program to facilitate associate professors’ promotion to full professor, paying particular attention to female associate professors in STEM disciplines.
		The authors initiated the study in 2008 with a comprehensive needs assessment of University of North Carolina-Charlotte associate professors’ attitudes toward and perceptions of the promotion process. Based on the responses, they developed a series of peer and vertical mentoring programs, including a program focused on female associate professors in STEM disciplines. Two years after launching the needs assessment, the authors administered a follow-up survey to gauge the program’s progress.
		The initial needs assessment found substantial gender differences in associate professors’ perceptions of the promotion process. Women were significantly more likely than men to believe that departmental standards for promotion were unclear, that their department chair did not provide career support or guidance, and that factors such as race and gender would influence the promotion process.
		Responses to the follow-up survey marked a significant increase in the proportion of associate professors reporting having a mentor and lessening of gender disparities regarding perceptions of the promotion process. Those who indicated that they had a mentor were significantly more likely than those who did not to perceive incentives for promotion and to believe that promotion criteria were clear.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, STEM professional",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Inquiry-based science and technology enrichment program: Green earth enhanced with inquiry and technology","Kim, H. (2011). Inquiry-based science and technology enrichment program: Green earth enhanced with inquiry and technology. Journal of Science Education & Technology, 20(6), 803-814.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:

The study’s objective was to examine the impact of the Inquiry-based Science and Technology Enrichment Program (InSTEP)—a week-long, half-day summer science curriculum for female 8th-grade students—on participants’ interest and confidence in studying science.
The author used a survey to compare students’ attitudes about science before and after participating in InSTEP.
The author found that among the female 8th-grade students who chose to enroll in InSTEP, interest in science increased, gender stereotypes about science and scientists were undermined, and overall attitudes toward science improved by the end of the program. There were no significant differences in students’ anxiety about taking science courses.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to InSTEP; other factors are likely to have contributed.","Inquiry-based Science and Technology Enrichment Program (InSTEP)","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Underneath it all: Gender role identification and women chemists’ career choices","Grunert, M., & Bodner, G. (2011). Underneath it all: Gender role identification and women chemists’ career choices. Science Education International, 22(4), 292-301.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This study examined the career perspectives and decisions of women earning doctorates in chemistry in the United States. Specifically, the study looked at the factors women considered when making career decisions, their perceptions of different careers, and the factors that motivated their career choices.
	The researchers conducted three interviews with each of 10 participants at two large research universities in the Midwest. All participants were one or two years away from completing their doctorate in chemistry. The study analyzed the data collected to develop a career decision-making model for women in chemistry doctorate programs.
	The study found that most participants made career decisions based on their perceptions of the lifestyle required to be successful. Most participants believed that a tenure-track position in a chemistry department was at odds with the traditional female role of primary caretaker in the home. Both single and married women believed that pursuing a research-intensive career would require sacrificing time with family and delaying having children, whereas the sole lesbian participant stated that she did not feel confined to a traditional female family role and was not opposed to the heavy work requirement of a tenure-track university position.
	The study also found that women perceived female faculty in their departments to exhibit masculine traits, and believed that it would be necessary to adopt a more masculine personality to be respected and successful in a chemistry department.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Surprising possibilities imagined and realized through information technology: Encouraging high school girls’ interests in information technology","Forssen, A., Lauriski-Karriker, T., Harriger, A., & Moskal, B. (2011). Surprising possibilities imagined and realized through information technology: Encouraging high school girls’ interests in information technology. Journal of STEM Education: Innovations and Research, 12(5-6), 46-57.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study examined the overall and gender-specific impact of the Surprising Possibilities Imagined and Realized through Information Technology (SPIRIT) program on high school students’ attitudes toward information technology (IT), with special focus on increasing female students’ confidence and abilities in using technology. The SPIRIT program, which took place during two weeks in summer 2009 at a large Midwestern university, had female and male students use a three-dimensional programming environment called Alice to create animations for storytelling. Participants also completed other hands-on activities and heard presentations from professionals on the use of IT in their jobs.
		The authors administered 20-item baseline and follow-up surveys to students on general attitudes toward and gender stereotypes in IT and compared responses before and after the SPIRIT program. Of the76 students who attended the program, 74 participated in the study. Because the program aimed to increase girls’ interests in IT, the study sample was 70 percent female.
		The study found that, after attending the SPIRIT program, female students demonstrated no change in general attitudes toward IT, but showed a significant reduction in believing the gender stereotype that women do not perform as well as men in IT.
		In looking at emergent themes of open-ended survey responses, the authors found that women’s attitudes toward the characteristics of a person with a career in IT changed from a focus on the person’s skills or intelligence before participation in SPIRIT to a more diverse description of characteristics after participation in the program. Similarly, female students provided a more expansive set of example IT careers after participation in SPIRIT. The authors did not conduct statistical tests on these differences.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering, and mathematics","Ong, M., Wright, C., Espinosa, L., & Orfield, G. (2011). Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering, and mathematics. Harvard Educational Review, 81(2), 172-208.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study examined existing empirical research on the challenges that minority women have faced in science, technology, engineering, and mathematics (STEM) disciplines at the undergraduate and graduate levels, emphasizing factors that promote or discourage retention and transition to employment or further study.
	The authors analyzed 116 empirical studies conducted from 1970 to 2008 that examine aspects of the academic and career experiences of minority women in STEM disciplines, identifying the findings of each study and synthesizing these features to pinpoint trends and opportunities for further investigation.
	The authors summarized the studies’ conclusions regarding the effects of academic climate, enrichment programs, self-confidence, and relationships with faculty, peers, and family on minority women’s persistence in undergraduate STEM majors, highlighting lack of recognition from peers and faculty members and complex family expectations and dynamics as factors that hinder them from completing STEM degrees.
	At the graduate level, many of the same influences, particularly family expectations and fraught relationships with peers and professors, present challenges for minority women in STEM disciplines. Academic and social transitions from minority-serving institutions to predominantly white institutions and the financial demands of graduate study also represent cross-cutting obstacles.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM)","Stout, J., Dasgupta, N., Hunsinger, M., & McManus, M. (2011). STEMing the tide: Using in-group experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100, 255-270. [one of three studies described in a single report]","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Low-Unfavorable impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to examine the impact of reading a paragraph-long biography of a female engineer on female engineering majors’ implicit and explicit attitudes toward math and English, their self-efficacy in engineering, their identification with the engineers in the assigned text, and their intention to pursue a career in engineering.
The study used computerized tasks to measure outcomes for female students who read a paragraph-long biography of a female engineer compared with those of female students who read either (1) biographies of male engineers or (2) a description of engineering innovations with no mention of the engineer’s gender.
The study found that attitudes of female students who read the female biography did not favor math over English or vice versa, whereas those who read the male biography or the description of engineering innovations demonstrated negative implicit attitudes toward math and a preference for English over math.
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to reading a paragraph-long biography about female engineers, and not to other factors.",,"Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Residential treatment for homeless female veterans with psychiatric and substance use disorders: Effect on 1-year clinical outcomes","Harpaz-Rotem, I., Rosenheck, R. A., & Desai, R. (2011). Residential treatment for homeless female veterans with psychiatric and substance use disorders: Effect on 1-year clinical outcomes. Journal of Rehabilitation Research & Development, 48(8), 891-899.",Veterans,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

	The study examined the impact of community residential treatment on female veterans’ employment and health outcomes.
	The study team used statistical tests to examine the differences between two groups: (1) people who were offered at least 30 or more days of residential treatment (RT) services at Veterans Affairs (VA) medical centers and (2) people who were not offered these services or were offered these services for less than 30 days. The data for this study came from case managers, discharge summaries, interviews, and standardized self-reported measures.
	The study showed that receiving RT services was associated with a greater number of days worked but with mixed results on health outcomes.
	The quality of causal evidence presented in this report is moderate because it is based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the RT program, but other factors might also have contributed to the effects.","Residential Treatment","Health Veterans' reemployment","Female, Veteran or military",,"United States",2011,https://www.researchgate.net/profile/Ilan_Harpaz-Rotem/publication/51784250_Res…,"Employment and Training Review Protocol"
"What predicts middle school girls’ interest in computing?","Denner, J. (2011). What predicts middle school girls’ interest in computing? International Journal of Gender, Science and Technology, 3(1), 54-69.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This paper analyzed white and Latina middle school girls’ interest in and knowledge of careers in computer science following their participation in a voluntary after-school and summer school program in central California.
	The author administered a survey to 140 program participants and conducted stepwise multiple regressions to identify factors that significantly affected the girls’ interest in computing classes and careers. The author also ran secondary analyses to address possible indirect or mediating influences on the girls’ interest in computing classes and careers.
	The study found that three factors significantly predicted girls’ interest in computing classes and careers: (1) a youth’s level of technological curiosity, (2) perceived support from school peers and teachers to pursue computing classes and degrees, and (3) feelings that computers were relevant to her life.
	The study also found that although perceived parental support and gender stereotypes did not directly predict the girls’ interest in computing classes and careers, these factors did indirectly predict girls’ interest in computing.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female, White, Hispanic of any race",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Recruitment strategies for gender equity: Lessons from cohort 1 and cohort 2 advance institutions","Zajicek, A., Morimoto, S., Terdalkar, A., Hunt, V., Rencis, J., & Lisnic, R. (2011). Recruitment strategies for gender equity: Lessons from cohort 1 and cohort 2 advance institutions. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts
      
    
  
              


      
            Employment","Summary:


The study’s objective was to evaluate the effectiveness of the ADVANCE institutional transformation program (ADVANCE-IT), sponsored by the National Science Foundation (NSF), on the recruitment of female engineering faculty at participating U.S. colleges and universities.
The authors used program implementation and outcome data obtained from seven participating colleges and universities to compare the number of female engineering faculty before and after institutions participated in the intervention.
The study found no statistically significant relationships between participation in the ADVANCE-IT program and the number of female assistant professors at those institutions.
The quality of causal evidence presented in this study is low because the authors conducted no tests of statistical significance and did not observe outcomes for multiple periods before the intervention. This means we are not confident that the estimated effects would have been attributable to the ADVANCE-IT program; other factors are likely to have contributed.","NSF’s ADVANCE-IT Program","Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, STEM professional",,"United States",2011,http://www.asee.org/file_server/papers/attachment/file/0001/1532/Recruitment_St…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Evolution of a faculty mentoring program for STEM women","Nemiro, J., Hacker, B., Tucker, S., Ferrel, M., Prall, D., & Dejonghe, E. (2011). Evolution of a faculty mentoring program for STEM women. International Journal of Gender, Science and Technology, 3(3), 644-658.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	This case study described implementation of a STEM mentoring program for female faculty at a polytechnic institution, including sucesses, challenges, and lessons learned.
	After each of three phases of program implementation, study authors administered participant surveys to capture satisfaction and usefulness of the program. They also accessed administrative data on current employment status of female STEM faculty at the institution.
	The surveys reflected that 80 percent of participants were satisfied with the program, finding it valuable for networking with colleagues, accessing department or college resources, and gaining advice on teaching strategies and achieving a work/life balance.
	A preliminary look at teacher retention showed program participants staying at the college at a higher rate than female STEM faculty who did not participate in the mentoring program.",,"Mentoring Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Establishing the foundation for future organizational reform and transformation at a large private university to expand the representation of women faculty","Bailey, M., Marchetti, C., DeBartolo, E., Mozrall, J., Williams, G., Baum, S., & LaLonde, S. (2011). Establishing the foundation for future organizational reform and transformation at a large private university to expand the representation of women faculty. Proceedings of the 2011 American Society for Engineering Education Annual Conference & Exposition, Washington, DC.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		This paper described the ADVANCE IT-Catalyst study that examined the barriers facing female STEM faculty members at six Rochester Institute of Technology (RIT) colleges, compared with men at RIT and women at similar universities. The research examined faculty rank, tenure, career track, leadership opportunities, and salary, and aimed to inform RIT’s efforts to recruit, retain, and advance female STEM faculty.
		The authors used institutional records from 2004 to 2010 to measure trends in recruitment, retention, advancement, and compensation, by faculty gender. A career-life survey administered to gather information on STEM faculty’s teaching and research experiences, including career satisfaction and work-life balance, complemented the institutional data. The authors tested for statistically significant differences in responses between men and women. Finally, the study benchmarked university policies and survey results against policies and survey results at other similar universities.
		In terms of retention, the study found that female faculty hired from 2002 to 2009 were twice as likely to have left RIT by 2010 than male faculty and, overall, the institution had a significantly lower percentage of female science and engineering tenured and tenure-track faculty in 2010 than the national average.
		In terms of leadership advancement and salary, the percentage of STEM leadership positions held by women in 2010 was proportional to the percentage of STEM faculty who were women. The study found that salary adjustments made by the university during the study period reduced the salary gap for female assistant and associate professors, but actually increased the gap for full professors. In the survey, women reported having to exert more effort and make more personal sacrifices to advance their careers than men, though most women felt that the tenure process was fair. Finally, the benchmarking study of policies and benefits found that RIT lagged behind other universities in terms of tenure clock extension, mentoring, and maternity leave.",,"Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Female, STEM professional",,"United States",2011,http://www.asee.org/public/conferences/1/papers/2271/download,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The role of living–learning programs in women’s plans to attend graduate school in STEM fields","Szelényi, K., & Inkelas, K. (2011). The role of living–learning programs in women’s plans to attend graduate school in STEM fields. Research in Higher Education, 52(4), 349-369.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to assess the impact of participating in a female-only STEM living-learning program on undergraduate female STEM majors’ intentions to pursue graduate studies in a STEM field.
The authors conducted regression analysis using data from the 2004–2007 National Study of Living Learning Programs (NSLLP) to determine if women who participated in female-only STEM living-learning programs were more likely than women in other living arrangements to express plans to attend graduate school in a STEM field.
The study found that the female undergraduate STEM majors living in a female-only STEM living-learning community were more likely to have plans to attend graduate school in a STEM field than female STEM majors living in coeducational STEM living-learning communities, non-STEM living-learning communities, or traditional residence halls.
The quality of causal evidence presented in this report is low because the authors’ analysis did not account for important individual characteristics related to the outcome that could differ across groups. This means we are not confident that the estimated effects are attributable to female-only STEM living-learning programs. Other factors are likely to have contributed.","STEM Living-Learning Programs","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Pipelines and pathways: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence","Espinosa, L. (2011). Pipelines and pathways: Women of color in undergraduate STEM majors and the college experiences that contribute to persistence. Harvard Educational Review, 81(2), 209-240.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to examine how undergraduate women of color’s precollege characteristics and college experiences and the characteristics of their undergraduate institutions affected their persistence in STEM majors throughout college compared with white women.
		The author used data from the Higher Education Research Institute Cooperative Institutional Research Program at the University of California, Los Angeles, to model whether college experiences mediated some of the effect of precollege characteristics and institutional settings on persistence in STEM majors, as hypothesized. Survey data were collected when students entered college in 2004 and again in the spring of 2008, during students’ fourth year of college. The study included a sample of 1,250 women of color and 891 white women at four-year colleges and universities across the United States who were first-time, full-time students. Results were estimated separately for women of color and white women.
		The study found that both white women and women of color persisted in STEM at a rate of 57 percent from the time they stated an intent to major in STEM at college entrance to the spring semester of their fourth year, and that high school grade point average was a significant predictor of persistence for both groups. As hypothesized, college experiences had a significant effect on the persistence of women of color in STEM: those who found satisfaction with their curriculum, engaged in peer discussion on course content outside the classroom, joined a major-related club, tutored another student, participated in research programs, and intended to major in engineering were more likely to persist in a STEM major; the same factors were significant for white women, with the exception of joining a major-related club. In addition, women of color who attended a private college and an institution with a higher percentage of students majoring in STEM were more likely to stay in a STEM major; the opposite was true for women of color who attended a highly selective institution. For white women, these institutional factors were not significant predictors of persistence in STEM.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Who wants to have a career in science or math? Exploring adolescents’ future aspirations by gender and race/ethnicity","Riegle-Crumb, C., Moore, C., & Ramos-Wada, A. (2011). Who wants to have a career in science or math? Exploring adolescents’ future aspirations by gender and race/ethnicity. Science Education, 95(3), 458-476.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The study's objective was to examine whether different gender and racial/ethnic subgroups of 8th-grade students in the United States varied in their aspirations of pursuing a career in science or math. The study further sought to observe how students’ attitudes toward and achievement in science and math helped explain any disparities in career aspirations by gender and race/ethnicity.
	The authors conducted a regression analysis that estimated the likelihood of different gender and race/ethnicity subgroups stating that they “would like a job that involves using science/math,” controlling for their attitudes toward and achievement in science/math. Career aspirations for groups traditionally underrepresented in science, technology, engineering, and mathematics—including white, black, and Hispanic girls and black and Hispanic boys—were compared to those of white boys. The study used data from the 2003 Trends in International Mathematics and Science Study, specifically looking at a nationally representative sample of 8th-grade students in the United States.
	In the primary analysis of science aspirations, no female and/or racial minorities had a statistically different likelihood of aspiring to a career in science than white boys. Black boys had comparable career aspirations to white boys even before controlling for any other factors, and Hispanic and white boys were similar after controlling for socioeconomic factors. Both white and Hispanic girls had similar career aspirations to white boys after controlling for enjoyment of science.
	In the primary analysis of math aspirations, all female subgroups were significantly less likely to aspire to a math career than white boys, even after controlling for general school attitudes, test scores, enjoyment in the subject, and belief in their own math ability. Black and Hispanic boys were as likely as white boys to be interested in pursuing a career in math before controlling for any of those factors.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Youth, Other barriers, Female, Male",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Programs for undergraduate women in science and engineering: Issues, problems, and solutions","Fox, M., Sonnert, G., & Nikiforova, I. (2011). Programs for undergraduate women in science and engineering: Issues, problems, and solutions. Gender & Society, 25(5), 589-615.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The authors summarized the goals, concerns, and activities of undergraduate programs for women in science and engineering. They focused on whether the programs considered individual-level concerns, such as self-esteem and academic ability, or structural barriers, such as faculty, campus, and administrative attitudes toward women, as the primary obstacles facing women in these fields.
	In 2002, the authors surveyed the directors of undergraduate programs for women in science and engineering, obtaining 38 responses out of 48 programs nationwide. They analyzed the survey responses by translating rating scales into mean scores and comparing those means using paired sample t-tests.
	The study found that directors of undergraduate programs for women in science and engineering generally conceived of the central problem facing their students as structural rather than individual. The activities these programs provided, however, focused predominantly on shoring up individual skills and self-confidence—for instance through social gatherings and links to support services—rather than addressing the structural issues they viewed as paramount.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2011,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Retiree health insurance and disengagement from a career job.","Robinson, C., & Clark, R. (2010). Retiree health insurance and disengagement from a career job. Journal of Labor Research, 31(3), 247-262.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-No impacts
      
    
  
              


      
            Employment","Summary:

The study examined the effect of availability of retiree health insurance (RHI) on a person’s decision to leave a career job (a measure of retirement).
Using data from the Health and Retirement Study (HRS), the authors used a nonexperimental approach to estimate the likelihood of retirement for those who worked at firms with RHI compared with those who worked at firms without RHI.
The study found that workers at firms that offer RHI are more likely to retire than those at firms that do not offer RHI.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to the availability at RHI, but other factors might also have contributed.","Retiree Health Insurance","Other health and safety Older workers' programs","Adult, Older worker",,"United States",2010,https://link.springer.com/article/10.1007/s12122-010-9091-4,"Employment and Training Review Protocol"
"The Effectiveness of Institutional Intervention on Minimizing Demographic Inertia and Improving the Representation of Women Faculty in Higher Education","Bakian, A.V., & Sullivan, K.A. (2010). The Effectiveness of Institutional Intervention on Minimizing Demographic Inertia and Improving the Representation of Women Faculty in Higher Education. International Journal of Gender, Science and Technology, 2(2), 207-234.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

This study’s objective was to examine the effect of the ADVANCE program sponsored by the National Science Foundation (NSF) on women’s representation among tenured and tenure-track faculty.
The study took place at one unnamed research university. The authors compared the transition probabilities (that is, the probabilities of being hired, retained, or promoted) for men and women, before and during the period ADVANCE was in effect.
The study found that the ADVANCE program enhanced women’s representation among faculty in science and engineering departments.
The quality of causal evidence presented in this study is low. This means we are not confident that the estimated effects are attributable to the ADVANCE program. Other factors are likely to have contributed.","the ADVANCE Program","Other disparities or discrimination in employment and earnings Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2010,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Change the gender composition of high school computing courses (case study 2): Attracting females and minority students through targeted recruiting","Cohoon, J.M. (2010). Change the gender composition of high school computing courses (case study 2): Attracting females and minority students through targeted recruiting. Boulder, CO: National Center for Women & Information Technology.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to describe several critical components to targeted recruitment for information technology courses, with a particular focus on the recruitment of girls and women.
		The author described the strategies used by one high school computer science teacher to increase the enrollment of girls in his Advanced Placement Computer Science course. The author then suggested several components that are necessary for any targeted recruitment strategy to be successful.
		The study recommended having a strategic recruiting plan that outlined the goal, partners, evaluation plan, and materials of the recruiting effort. The goal should be specific and quantifiable, and partnerships should be based on existing contacts who work with the target audience. The recruitment message should focus on the interests and concerns of the target audience, and should be conveyed in multiple ways and delivered through a credible source. The study summarized these recommendations in a recruiting quick list.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2010,http://www.ncwit.org/sites/default/files/resources/changegendercompositionhighs…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Prisoner reentry experiences of adult females: Characteristics, Service Receipt, and Outcomes of Participants in the SVORI multi-site evaluation.","Lindquist, C. H., Lattimore, K. B., & Visher C. A. (2009) Prisoner reentry experiences of adult females: Characteristics, Service Receipt, and Outcomes of Participants in the SVORI multi-site evaluation. Research Triangle Park, NC: RTI International.",Reentry,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-No impacts
      
    
  
              


      
            Public benefit receipt","Summary:

	The study’s objective was to examine the impact of a violent prisoner reentry program, the Serious and Violent Offender Reentry Initiative (SVORI), on several outcomes of female prisoners reentering society, including employment, public benefit receipt, and recidivism.
	The authors used a matched comparison group design, using propensity-score weights, to analyze impacts of the SVORI program for females released from prison. The authors collected four waves of survey data to analyze impacts from before release up to 15 months after release. In addition, the authors obtained state administrative data describing recidivism at 24 months after release.
	The study found that people in the SVORI group were more likely to be employed after release than those in the comparison group, but it found no significant differences between the groups in public benefit receipt or recidivism outcomes.
	The quality of causal evidence presented in this report for the recidivism outcomes based on survey data is moderate because it was based on a well-implemented nonexperimental design. This means we are confident that any estimated effects would have been attributable to SVORI and not to other factors had the study found statistically significant effects. The quality of causal evidence presented in this report for the recidivism outcomes based on administrative data and for the employment and public benefit receipt outcomes is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects on employment are attributable to SVORI; other factors are likely to have contributed.","the Serious and Violent Offender Reentry Initiative (SVORI)",Reentry,"Justice-involved, Female",,"United States",2010,https://www.ncjrs.gov/App/AbstractDB/AbstractDBDetails.aspx?id=252453,"Employment and Training Review Protocol"
"Framing Social Security Reform: Behavioral responses to changes in the full retirement age","Behaghel, L., & Blau, D. M. (2010) Framing Social Security Reform: Behavioral responses to changes in the full retirement age. (IZA Discussion Paper No. 5310). Bonn, Germany: IZA.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of changes to the Full Retirement Age (FRA) on labor force exit, Old Age and Survivor’s Insurance (OASI) claims, and retirement.
The study uses a statistical model and the data from the Health and Retirement Study (HRS) to compare outcomes of people in birth cohorts who were affected by the change in FRA with those of a comparison group of people who were not.
The study found that the change in the FRA was associated with significant increases in the likelihood of claiming OASI and exiting the labor force at the new, later full retirement age.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the change in FRA, but other factors might also have contributed.","changes to the Full Retirement Age (FRA)","Older workers' programs Other employment and reemployment","Older worker",,"United States",2010,https://www.aeaweb.org/articles?id=10.1257/pol.4.4.41,"Employment and Training Review Protocol"
"Reducing the gender achievement gap in college science: A classroom study of values affirmation","Miyake, A., Kost-Smith, L., Finkelstein, N., Pollock, S., Cohen, G., & Ito, T. (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330 (6008), 1234–1237.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:


The study’s objective was to assess the effect of values affirmation on female students’ performance in an introductory college physics course.
Students in the course were randomly assigned to either a treatment group, in which they completed a writing exercise affirming their personal values, or a control group, in which they described values that might be important to others. The authors collected exam scores and course grades for students in both study groups.
The study found that, although male students’ exam scores remained higher than female students’ scores, the gender gap was significantly smaller for students in the values affirmation group than for students in the control group.
The quality of causal evidence presented in this report is low because it was a randomized controlled trial with high attrition that did not control for background characteristics likely to have affected the outcomes of interest. This means we are not confident that the estimated effects are attributable to values affirmation; other factors are likely to have contributed.","Values Affirmation","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2010,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Persistence of women and minorities in STEM field majors: Is it the school that matters?","Griffith, A. (2010). Persistence of women and minorities in STEM field majors: Is it the school that matters? Economics of Education Review, 29, 911-922.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study examined institutional and student factors that had the greatest effect on students’ decisions to persist in STEM majors, emphasizing factors promoting STEM degree completion for female and minority students.
		The author analyzed STEM persistence to the fourth year of undergraduate study in two data sets: the 1988 National Education Longitudinal Study and the 1999 National Longitudinal Survey of Freshmen (NLSF). In each data set, the author separately examined factors that predicted STEM degree completion for male, female, minority, and nonminority students who indicated at matriculation that they intended to pursue STEM majors.
		Students’ academic backgrounds were most likely to predict persistence in STEM. For both women and minorities in both data sets, a higher ratio of first-year STEM grade point average to total first-year grade point average was associated with a statistically significantly higher probability of completing a STEM degree. For minority students in the NLSF sample, taking more STEM Advanced Placement courses in high school also promoted persistence in postsecondary STEM study.
		No institutional factors were statistically significant predictors of STEM degree completion for either women or minorities.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2010,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Can Social Security explain trends in labor force participation of older men in the United States?","Blau, D. M., & Goodstein, R. M. (2010). Can Social Security explain trends in labor force participation of older men in the United States? Journal of Human Resources, 45(2), 328-363.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of changes in Social Security Administration (SSA) retirement rules on men’s labor force participation rates in the 1960s–1980s and 1990s–2000s.
The study used regression analyses to predicted labor force participation rates using actual and counterfactual SSA retirement rules. These analyses use data from the Current Population Survey (CPS) and the Survey of Income and Program Participation (SIPP) for the period from 1962 to 2005 merged with data from SSA on taxable earnings.
The study found that changes in the delayed retirement credit and full retirement age in 1983 could explain one-quarter to one-half of the increase in labor force participation in the early 2000s.
The quality of causal evidence presented in this report is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to SSA reforms, but other factors might also have contributed.","Social Security Administration Reforms","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2010,http://jhr.uwpress.org/content/45/2/328.abstract,"Employment and Training Review Protocol"
"Climate for retention to graduation: A mixed methods investigation of student perceptions of engineering departments and programs","Wao, H., Lee, R., & Borman, K. (2010). Climate for retention to graduation: A mixed methods investigation of student perceptions of engineering departments and programs. Journal of Women and Minorities in Science and Engineering, 16(4), 293-317.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The study's objective was to determine what characteristics of a college department (referred to as the climate) were conducive to retaining women and underrepresented minorities enrolled in an engineering program through graduation. The study took place at four engineering programs in Florida universities during the 2007–2008 academic year.
		The authors administered a Likert scale survey to a convenience sample of undergraduate students enrolled in engineering programs that measured nine aspects of the department’s climate (for example, faculty involvement or support, diversity, and student integration with department) as well as the student’s intent to leave the program. The authors then used regression analysis to determine which of the nine aspects of climate predicted a student’s intent to leave the program. In-person interviews and six focus groups supplemented survey findings.
		The study found that the aspects of college department climate predictive of a student’s intent to leave the program were institutional support (or the level of support and services provided by the institution to help students succeed), social and academic fit (or how well the student felt he or she was a part of the department), and personal agency and peer support (or one’s own involvement in the learning process and a peer atmosphere that encouraged success). There were no statistically significant differences in findings by gender or ethnicity, although interview and focus group data point toward complexities in how women and minorities experienced climate, suggesting that they might be more inclined than majority groups to rely on social support and fit than institutional support.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs","Youth, Other barriers, Female",,"United States",2010,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Why so few? Women in science, technology, engineering, and mathematics","Hill, C., Corbett, C., & St. Rose, A. (2010). Why so few? Women in science, technology, engineering, and mathematics. Washington, D.C.: American Association of University Women.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		This publication summarized the obstacles facing girls and women who wish to pursue academic or professional careers in STEM fields and highlighted solutions to these challenges at the secondary, postsecondary, and professional levels.
		The authors presented scholars’ investigations into issues surrounding gender differences in interest, retention, or careers in STEM, drawing on both interviews with the profiled scholars and their published studies.
		The paper concluded that bias and stereotypes, whether implicit or explicit, hindered girls’ development in STEM. According to the paper, promoting the idea that intelligence is not fixed and innate but can expand through hard work, dubbed the growth mindset, combats the notion that boys are inherently better at math and science than girls and encourages girls to persevere.
		Further, women in academic and industry STEM positions continued to suffer from bias and stereotypes, which can manifest themselves through unsupportive atmospheres or distorted perceptions of female employees’ competence. Clear standards for advancement can counteract some of these tendencies.",,"Youth programs Science, Technology, Engineering, and Math (STEM) programs Other disparities or discrimination in employment and earnings","Youth, Female",,"United States",2010,http://www.aauw.org/files/2013/02/Why-So-Few-Women-in-Science-Technology-Engine…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Elderly immigrants’ labor supply response to supplemental security income.","Kaushal, N. (2010). Elderly immigrants’ labor supply response to supplemental security income. Journal of Policy Analysis and Management, 29(1), 137-162.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study objective is to examine the impact of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 on employment outcomes of foreign-born people and noncitizens ages 60 to 74
The study used a statistical model and the data from the Current Population Survey’s monthly outgoing rotation group and the March Current Population Survey from 1994 to 2005 to estimate the impact.
The study found that the PRWORA is associated with increased employment and decreased retirement for foreign-born men compared with U.S.-born men, but it found no statistically significant association with the employment for foreign-born women.
The quality of the causal evidence presented in this report is low because the study is a nonexperimental analysis that does not demonstrate that the groups were similar before the policy change. This means we are not confident that the estimated effects are attributable to PRWORA; other factors are likely to have contributed.","Personal Responsibility and Work Opportunity Reconciliation Act of 1996","Federal retirement benefits Foreign and temporary workers","Adult, Older worker, Immigrant",,"United States",2010,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4222576/,"Employment and Training Review Protocol"
"Women in the academy: Female leadership in STEM education and the evolution of a mentoring web","Gorman, S., Durmowicz, M., Roskes, E., & Slattery, S. (2010). Women in the academy: Female leadership in STEM education and the evolution of a mentoring web. Forum on Public Policy Online, 2010(2), 1-21.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	
	
		The authors described the Stevenson University School of the Sciences in Maryland as an example of an academic institution that deviated widely from the norm of female under-representation in STEM departments, not only among enrolled students but among faculty members and departmental leadership. They proposed several institutional mechanisms for this unusual success, including mentoring programs for several groups: academic leadership, faculty seeking promotion, new faculty, and students.
		At the time of publication, two School of the Sciences faculty members had applied for promotion under the Faculty Mentoring and Evaluation Committee system, a formalized mentoring structure through which each faculty member received support and professional guidance from a mentor group comprising the mentee’s department chair and at least two faculty peers, one from outside the department. Both faculty members were successful.
		Preliminary assessments of student mentoring programs that paired freshman biology or chemistry majors with upper-class students in their department suggest that participation augmented both mentors’ and mentees’ sense of community, support, and encouragement. In some cases, the program also promoted retention in the discipline.",,"Youth programs Mentoring Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2010,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The effects of changes in state SSI supplements on preretirement labor supply","Neumark, D., & Powers, E. T. (2005). The effects of changes in state SSI supplements on preretirement labor supply. Public Finance Review, 33(1), 3-35.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of a theoretical increase in Supplemental Security Income (SSI) benefits on the preretirement employment outcomes for likely SSI participants ages 60 to 64.
Using data from the annual Current Population Survey from 1979 to 2001 and state-level SSI receipt data, the authors used regression methods in a nonexperimental analysis of likely SSI participants to compare the outcomes of workers ages 62 to 64 with those of workers ages 60 to 61.
The study found that an increase in monthly SSI benefits was associated with a statistically significant decrease in the employment rate and number of hours worked for likely SSI recipients ages 62 to 64 compared with those ages 60 to 61.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to changes in SSI benefits; other factors are likely to have contributed.","Supplemental Security Income (SSI) benefits","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2010,https://journals.sagepub.com/doi/pdf/10.1177/1091142104270655,"Employment and Training Review Protocol"
"The influences on bridge employment decisions.","Pengcharoen, C., & Shultz, K. S. (2010). The influences on bridge employment decisions. International Journal of Manpower, 31(3), 322-336. doi:10.1108/01437721011050602","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of work schedule flexibility on retirement outcomes among older workers, where work schedule flexibility was measured as the ability to reduce hours from one’s regular work schedule.
The study used a statistical model (multinomial logit regression) and the longitudinal data from the Health and Retirement Study (HRS) from 1992 and 2002 to examine the relationship between reporting work schedule flexibility or inflexibility during the baseline survey and reporting complete retirement, partial retirement, or continued employment at the follow-up survey 10 years later among older workers.
The study found that individuals who reported work schedule inflexibility in the 1992 were more likely to report completely retired than partially retired in 2002, but when comparing the likelihood of reporting continued employment to complete or partial retirement, there was no difference between older workers reported work schedule flexibility and those who reported inflexibility in 1992.
The quality of causal evidence presented in this study is low because the authors did not demonstrate that the participants reporting work schedule flexibility were similar to the participants reporting work schedule inflexibility in 1992. This means we are not confident that the estimated effects are attributable to work schedule flexibility; other factors are likely to have contributed.",,"Other wages and benefits","Adult, Older worker",,"United States",2010,https://www.researchgate.net/publication/46545640_The_influences_on_bridge_empl…,"Employment and Training Review Protocol"
"Using appreciative inquiry as a tool to instigate transformational change in recruiting and developing women faculty in STEM disciplines","Nemiro, J., Hacker, B., Ferrel, M., & Guthrie, R. (2009). Using appreciative inquiry as a tool to instigate transformational change in recruiting and developing women faculty in STEM disciplines. International Journal of Gender, Science and Technology, 1(1), 5-35.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary:

	The authors’ main objective was to develop recommendations for recruiting and developing female faculty in STEM disciplines.
	The authors conducted a series of eight focus groups with science and engineering faculty at a university in the United States to discuss what the organization currently did well and what it could do well in the future, and drew out common themes across the discussions.
	The authors recommended the following best practices for institutions to better recruit female faculty in STEM: (1) create clear and transparent job postings; (2) promote the benefits that the department, university, and community have to offer potential candidates; and (3) actively recruit candidates (for example, by developing relationships with universities and pursuing candidates before they apply for a position).
	The authors also suggested best practices for developing female faculty, including training staff at all levels on the cognitive biases against women, creating within-organization mentoring and networking opportunities for female faculty in STEM, developing family-friendly policies such as leave and flexible schedules, allotting time and funding for professional development, and actively developing women for leadership positions.",,"Science, Technology, Engineering, and Math (STEM) programs","Female, STEM professional",,"United States",2009,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Supporting young women to enter engineering: Long-term effects of a middle school engineering outreach program for girls","Demetry, C., Hubelbank, J., Blaisdell, S., Sontgerath, S., Nicholson, M.E., Rosenthal, E., & Quinn, P. (2009). Supporting young women to enter engineering: Long-term effects of a middle school engineering outreach program for girls. Journal of Women and Minorities in Science and Engineering 15, 119-142.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to assess the long-term impact of Camp Reach, a summer engineering enrichment program for middle school girls, on enrollment in STEM courses in high school and college.
Admission to Camp Reach was determined through a lottery of girls who applied to the program and met eligibility requirements. Six to seven years later, the authors administered a survey to 109 girls who had been admitted to the program and 107 girls who had not. Many girls in the control group attended other, similar programs.
The study found that members of the control group who had attended other summer science or engineering programs at Worcester Polytechnic Institute (WPI) were significantly more likely to have taken high school calculus than were members of the Camp Reach Partial group. There were no statistically significant differences on taking physics, computer science, other science and engineering courses, or planning to major in engineering in high school.
The quality of causal evidence presented in this report is low because the authors adjusted the research sample after random assignment and did not include adequate controls to ensure that the resulting groups were similar on all relevant attributes. This means we are not confident that the estimated effects were attributable to Camp Reach; other factors are likely to have contributed.","Camp Reach","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2009,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"New evidence on the labor supply effects of the Social Security earnings test.","Friedberg, L., & Webb, A. (2009). New evidence on the labor supply effects of the Social Security earnings test. Tax Policy and the Economy, 23(1), 1-36.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of changes to the Social Security earnings test in 1996 and 2000 on employment outcomes.
The authors used a nonexperimental design and the data from the Current Population Survey (CPS) and the Health and Retirement Study (HRS) to compare outcomes before and after the changes in earning test rules for those between the ages of 62 and 74 who faced different earnings test thresholds at different ages and years.
The study found that upcoming anticipated earnings test between ages 62 to 69 was associated with a significantly lower likelihood of having a job last week. A higher present value of upcoming anticipated earnings threshold between ages 62 to 69 was associated with a significantly higher likelihood of having a job last week.
The quality of causal evidence presented in this study is low because the study is a nonexperimental analysis that did not demonstrate that the groups being compared were similar and did not account for possible differences in the analysis. This means we are not confident that the estimated effects are attributable to the changes in the earnings test decision. Other factors are likely to have contributed.","Social Security Earnings Test","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2009,,"Employment and Training Review Protocol"
"Improving 24-month abstinence and employment outcomes for substance-dependent women receiving Temporary Assistance for Needy Families with intensive case management.","Morgenstern, J., Neighbors, C., Kuerbis, A., Riordan, A., Blanchard, K., McVeigh, K., Morgan, T., & McCrady, B. (2009). Improving 24-month abstinence and employment outcomes for substance-dependent women receiving Temporary Assistance for Needy Families with intensive case management. American Journal of Public Health, 99(2), 328-333.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact on employment of an intensive case management (ICM) program for substance-dependent women receiving Temporary Assistance for Needy Families (TANF).
Women identified as substance-dependent were randomly assigned to either the ICM or usual care treatment programs. The primary data sources were daily self-reports and follow-up interviews conducted 3, 9, 15, and 24 months after random assignment.
The study found that women in the ICM group were generally significantly more likely than women in the usual care group to be employed full-time, particularly at the 24-month follow-up period. In addition, the study found a significantly higher rate of improvement in employment outcomes for the ICM group compared with the usual care group.
The quality of causal evidence presented in this report is low because it was based on a randomized controlled trial with high attrition and the authors did not account for differences between the two groups in their analysis. This means we are not confident that the estimated effects are attributable to the ICM or usual care program; other factors are likely to have contributed.","Intensive Case Management (ICM) and Usual Care Programs","Other employment and reemployment Substance abuse recovery","Female, Low income",,"United States",2009,,"Employment and Training Review Protocol"
"Assessment of the impact of WorkSource job search services","Lee, K., Weeks, G., Bodeutsch, G., Clay-Poole, S., Garoflo, T., Petritz, M., Stromsdorfer, E., Field, J., Paterson, T., & Jennings, K. (2009). Assessment of the impact of WorkSource job search services. Washington: Washington State Employment Security Department Labor Market and Economic Analysis.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-Favorable impacts
      
    
  
              


      
            Public benefit receipt","Summary:

	The study’s objective was to assess the effectiveness of Washington State WorkSource job search services. WorkSource services included staff-assisted job matching, provision of labor market information, job search and placement assistance, and initial assessments.
	Using a nonexperimental design, the authors compared the employment and earnings of Unemployment Insurance (UI) claimants who received at least one WorkSource service with those of UI claimants who did not receive any WorkSource services using state UI administrative records for more than 64,000 Washington State UI claimants with no UI-reported wage earnings in the third quarter of 2007.
	The study found that WorkSource services were associated with a higher probability of employment and higher earnings growth for men and women who were intermittently (rather than continuously) employed in the baseline period.
	The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to WorkSource services. Other factors are likely to have contributed.","JSA services","RESEA Job search assistance and supportive services","Unemployed, Female, Male",,"United States",2009,https://esdorchardstorage.blob.core.windows.net/esdwa/Default/ESDWAGOV/labor-ma…,"Reemployment Review Protocol"
"The repeal of the retirement earnings test and the labor supply of older men.","Engelhardt, G. V., & Kumar, A. (2009). The repeal of the retirement earnings test and the labor supply of older men. Journal of Pension Economics & Finance, 8(4), 429-450.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of the repeal of the earnings test under the Senior Citizens Freedom to Work Act of 2000 on the labor supply and earnings of older men ages 62 to 72.
The authors used a nonexperimental approach and data from the Health and Retirement Study to compare labor force participation, annual hours of work, retirement behavior, and earnings of men who were and were not subject to the repeal of the earnings test.
The study found that the policy change was associated with a significant increase in older men’s average annual hours of work. The policy change was also associated with a change in the distribution of real earnings in that a smaller proportion of older men had earnings at or below the earnings test threshold.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000; other factors are likely to have contributed.","Senior Citizens Freedom to Work Act of 2000","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2009,,"Employment and Training Review Protocol"
"An empirical study of the effects of Social Security reforms on benefit claiming behavior and receipt using public-use administrative microdata.","Benítez-Silva, H., & Yin, N. (2009). An empirical study of the effects of Social Security reforms on benefit claiming behavior and receipt using public-use administrative microdata. Social Security Bulletin, 69(3), 77-95.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined the impact of three policy changes to the Social Security program that went into effect in 2000 on average monthly benefits and the proportion of new claimants by age.
The study was a nonexperimental analysis that examined outcomes for older workers by age and year using data from the 2004 Old Age, Survivors, and Disability Insurance program public-use microdata files.
The study found that the changes were associated with lower average monthly benefits for individuals ages 66 to 69 after the year 2000 relative to individuals of the same ages in 1994.
The quality of the causal evidence presented in this report is low because the authors did not demonstrate that the groups were similar before the intervention. This means we are not confident that the estimated effects are attributable to the intervention program; other factors are likely to have contributed.","Policy Changes to the Social Security Program","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2009,https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1480653,"Employment and Training Review Protocol"
"How did the elimination of the US earnings test above the normal retirement age affect labour supply expectations?","Michaud, P., & Van Soest, A. (2008). How did the elimination of the US earnings test above the normal retirement age affect labour supply expectations? Fiscal Studies, 29(2), 197-231. doi:10.1111/j.1475-5890.2008.00073.x","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The authors used a nonexperimental approach and the data from Health and Retirement Study data from 1996 to 2002 and Social Security earnings history to estimate the impact.
The study examined the impact of repealing the earnings test from Social Security recipients between full retirement age and age 70 on older workers’ subjective probability of retirement.
The authors used a nonexperimental approach and the data from Health and Retirement Study data from 1996 to 2002 and Social Security earnings history to estimate the impact.
The study found that after the repeal of the earnings test, the subjective probability of working of full-time past age 65 for older male workers who were likely to have most of their projected Social Security benefits reduced under the earnings test has increased their expected probability of working relative to those who were not likely to be affected. •The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design.This means we are somewhat confident that the estimated effects are attributable to the repeal of the earnings test, but other factors might also have contributed.","Social Security Earnings Test","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2008,,"Employment and Training Review Protocol"
"The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data.","Haider, S. J., & Loughran, D. S. (2008). The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data. Journal of Human Resources, 43(1), 57-87.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Mixed impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employer benefits receipt-Low-Favorable impacts
      
    
  
              


      
            Employer benefits receipt","Summary:

The study’s objective is to examine the impact of 1983 and 2000 policy changes that removed the Social Security earnings test for certain age groups on the earnings and employment outcomes of affected men
The study is a nonexperimental analysis that examines the correlations between the 1983 and 2000 policy changes and earnings and employment status. The study compares the outcomes of men affected by the policy changes with the outcomes of men of similar ages not affected by the policy changes.
The study finds that the 1983 policy change is associated with few changes to the observed outcomes and that the 2000 policy change is associated with higher earnings and hours worked per week for affected men.
The quality of causal evidence presented in this report is low because the study does not demonstrate that men affected by the policy changes are similar to the men not affected by the policy change, nor does it control for possible differences. This means we are not confident that the estimated effects are attributable to the 1983 and 2000 policy changes; other factors are likely to have contributed.","Social Security Earnings Test","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2008,http://jhr.uwpress.org/content/43/1/57.refs,"Employment and Training Review Protocol"
"State age protection laws and the Age Discrimination in Employment Act.","Lahey, J. State age protection laws and the Age Discrimination in Employment Act. (2008). The Journal of Law and Economics, 51(3), 433-460.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Low-No impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of enforcement of age discrimination laws on the employment and earnings of older male workers
The author used a nonexperimental regression model and data from the Current Population Survey to estimate the impact.
The study found that males older than 50 in states with age discrimination laws were 0.2 percentage points less likely to be hired compared with those in states without their own age discrimination laws.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the state laws on age discrimination, but other factors might also have contributed.","Age Discrimination Employment Act (ADEA)","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2008,https://www.jstor.org/stable/10.1086/589670?seq=1#page_scan_tab_contents,"Employment and Training Review Protocol"
"Supported employment for middle-aged and older people with schizophrenia.","Twamley, E., Narvaez, J., Becker, D., Bartels, S., & Jeste, D. (2008). Supported employment for middle-aged and older people with schizophrenia. American Journal of Psychiatric Rehabilitation, 11(1), 76-89.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the impact of Individual Placement and Support (IPS) on competitive employment for older people with schizophrenia
The study was a randomized controlled trial (RCT) and used weekly work logs cross-referenced with pay stubs to measure employment outcomes over a 12-month period.
The study found that IPS increased competitive employment and earnings compared with conventional services.
The quality of causal evidence presented in this report is low because the RCT has a confounding factor. This means we are not confident that the estimated effects are attributable to the IPS program; other factors are likely to have contributed.","Individual Placement and Support and Vocational Rehabilitation Program","Health Individuals facing barriers to employment Job search assistance and supportive services","Adult, Older worker, Disability",,"United States",2008,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2638571/,"Employment and Training Review Protocol"
"Workforce Investment Act non-experimental net impact evaluation final report","Heinrich, C., Mueser, P., Troske, K., & Benus, J. (2008). Workforce Investment Act non-experimental net impact evaluation final report. Columbia, MD: IMPAQ International, LLC.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages","Summary:

The report’s objective was to examine the effects of participation in the Workforce Investment Act (WIA) Adult and Dislocated Worker programs on average earnings and employment. Results for earnings of participants ages 22 to 26 were presented separately and are the focus of this review.
The effectiveness of the programs was evaluated using a matched comparison group design. Outcomes were examined up to 10 quarters after WIA participants entered the Adult or Dislocated Worker programs.
The study found that participants in the WIA Adult Program under age 26 earned significantly more than those in the comparison group every quarter after program entry. In the WIA Dislocated Worker program, participants under age 26 began earning significantly more than those in the comparison group a year following program entry.
The quality of the causal evidence presented in this study is moderate. This means we have confidence that the effects estimated in this study are attributable at least in part to WIA programs. However, as in any nonexperimental study, other factors not accounted for in the analysis might have contributed to the estimated effects.","the Workforce Investment Act (WIA)","Basic skills Community college education and other classroom training Job search assistance and supportive services Other training and education Work based and other occupational training","Adult, Youth, Dislocated or displaced worker, Female, Male",,"United States",2008,http://www.nawdp.org/Content/NavigationMenu/ResearchReports/2009-10-WIANon-Expe…,"Opportunities for Youth Review Protocol"
"Evaluating multi-treatment programs: Theory and evidence from the U.S. Job Training Partnership Act experiment.","Plesca, M., & Smith, J. (2007). Evaluating multi-treatment programs: Theory and evidence from the U.S. Job Training Partnership Act experiment. Empirical Economics, 32, 491-528.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Mod/high-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of the Job Training Partnership Act (JTPA) on earnings and employment for disadvantaged adults.
The authors conducted a randomized controlled trial. They analyzed data from one baseline survey, two follow-up surveys, and earnings and employment records.
The study found that men in the treatment group had earned more at the 30-month follow-up than men in the control group, and that women in the treatment group experienced significantly better earnings and employment outcomes at several points compared to the control group.
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to JTPA, and not to other factors.","Job Training Partnership Act","Other training and education Supported employment or other employment supports Job search assistance and supportive services","Adult, Female, Male, Low-skilled, Low income",,"United States",2007,,"Review Protocol"
"Labor market shocks and retirement: Do government programs matter?","Coile, C., & Levine, P. (2007). Labor market shocks and retirement: Do government programs matter? Journal of Public Economics, 91(10), 1902-1919.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Low-No impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined the impact of the generosity of unemployment insurance (UI) benefits on retirement decisions.
The study used a statistical model and data from the March Current Population Survey for 1980–2004 to estimate impacts.
The study found no relationship between the generosity of UI benefits and the likelihood of retirement or receipt of UI.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before changes in the generosity of UI benefits were observed. This means we are not confident that the estimated effects are attributable to changes in the generosity of UI benefits; other factors are likely to have contributed.","Unemployment Insurance (UI)","Unemployment Insurance","Adult, Older worker, Unemployed",,"United States",2007,http://academics.wellesley.edu/Economics/Coile/PDF%20files/coile-levine%20JPubE…,"Employment and Training Review Protocol"
"Effects of learning about gender discrimination on adolescent girls’ attitudes toward and interest in science","Weisgram, E., & Bigler, R. (2007). Effects of learning about gender discrimination on adolescent girls’ attitudes toward and interest in science. Psychology of Women Quarterly, 31, 262-269.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts
      
    
  
              


      
            Attitudes","Summary:

The study’s objective was to examine the impact of learning about gender discrimination in the sciences on girls’ attitudes toward and interest in science.
The study included middle school girls who attended a conference designed to improve their attitudes toward and interest in science. Participants were randomly assigned to the treatment condition, which included standard conference sessions on science careers and a session on gender discrimination in the sciences, or to the control group that attended only the standard sessions. The authors administered pre- and post-conference surveys to measure the impact of the intervention.
The study found significant increases in reported science self-efficacy and science utility value among the girls in the treatment condition, whereas girls in the control condition experienced a decrease in egalitarian attitudes toward science.
The quality of causal evidence provided in this study is high because it was a well-implemented randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the intervention and not to other factors.",,"Preventing discrimination Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2007,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Increasing retention of women engineering students","Sullivan, K., & Davis, R. (2007). Increasing retention of women engineering students. American Society for Engineering Education. Proceedings of the 2007 American Society for Engineering Education Annual Conference and Meeting. Washington, DC: American Society for Engineering Education.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-No impacts
      
    
  
              


      
            Attitudes
      


  
      
            Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to examine how participating in a study-designed program called the Jornada Intervention affected undergraduate female engineering students’ attitudes toward and persistence in studying engineering. Program activities included receiving a handheld computer and an invitation to participate in meetings to share experiences using the computer and suggestions for improving the user’s experience.
The study used a comparison group design to compare outcomes for the treatment group with those of women who enrolled in engineering in adjacent years. Data sources included an attitudinal survey administered for the first analysis and administrative records to measure students’ graduation rates for the second analysis.
In the first analysis, the study found no significant differences in attitudes toward engineering between the treatment and comparison groups. For the second analysis, though results were not presented and methods were not described, the authors reported that they uncovered significant differences in persistence between the treatment and comparison groups. They noted that program participants were significantly more likely to graduate in engineering than both (1) women from the same year who did not participate in the program and (2) women who enrolled in engineering in adjacent years.
The quality of causal evidence presented in this report is low because it does not include control variables to adjust for differences between treatment and comparison groups. This means we are not confident that the estimated effects are attributable to the intervention; other factors are likely to have contributed.","Jornada Intervention","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2007,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Long-term effects of a middle school engineering outreach program for girls: A controlled study.","Hubelbank, J., Demetry, C., Nicholson, S., Blaisdell, S., Quinn, P., Rosenthal, E., & Sontgerath, S. (2007). Long-term effects of a middle school engineering outreach program for girls: A controlled study. In Proceedings, American Society for Engineering Education Annual Conference & Exhibition.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to assess the long-term impact of Camp Reach, a summer engineering enrichment program for middle school girls, on enrollment in STEM courses in high school and college.
Admission to Camp Reach was determined through a lottery of girls who applied to the program and met eligibility requirements. Six to seven years later, the authors administered a survey to 88 girls who had been admitted to the program and 41 girls who had not.
The study found that Camp Reach attendees were significantly more likely to have taken computer science and other science or engineering courses in high school than those who did not attend.
The quality of causal evidence presented in this report is low because the authors adjusted the research sample after random assignment and did not include adequate controls to ensure that the resulting groups were similar on all relevant attributes. This means we are not confident that the estimated effects are attributable to Camp Reach; other factors are likely to have contributed.","Camp Reach","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2007,https://www.wpi.edu/Images/CMS/News/1106_LONG_TERM_EFFECTS_OF_A_MIDDLE_SCHOOL_E…,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"A tax on work for the elderly: Medicare as a secondary payer.","Goda, G. S., Shoven, J. B., & Slavov, S. N. (2007). A tax on work for the elderly: Medicare as a secondary payer. (NBER Working Paper No. 13383). Cambridge, MA: National Bureau of Economic Research.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the impact of Medicare as a Secondary Payer (MSP) on older workers’ labor force participation and full-time employment
The study used a nonexperimental regression design. The authors used data from the 1980 to 2006 March Supplement of the Current Population Survey.
The study found that labor force participation for older males increased following the enforcement of the MSP policy, and that older males and females in the labor force were more likely to be employed full time after the policy was enforced.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention, and confounding factors could have impacted older workers differentially during the time periods examined. This means we are not confident that the estimated effects are attributable to MSP; other factors are likely to have contributed.","Medicare as a Secondary Payer","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2007,https://www.nber.org/papers/w13383,"Employment and Training Review Protocol"
"New evidence on earnings and benefit claims following the changes in the retirement earnings test in 2000.","Song, J. G., & Manchester, J. (2006). New evidence on earnings and benefit claims following the changes in the retirement earnings test in 2000. (ORES Working Paper 107). Washington, DC: Social Security Administration, Office of Policy, Office of Research, Evaluation, and Statistics. Also available in Journal of Public Economics, 91(3–4), 669–700 (2007).","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-No impacts
      
    
  
              


      
            Employment
      


  
      
            Public benefits receipt-Mod/high-Unfavorable impacts
      
    
  
              


      
            Public benefit receipt","Summary:

The study examined the impact of the Senior Citizens Freedom to Work Act of 2000 on employment, earnings, and public benefit receipt outcomes of workers ages 65 to 69.
The study was a nonexperimental analysis and used Social Security Administrative data to estimate the impacts.
The study found that the Senior Citizens Freedom to Work Act of 2000 was associated with significant increases in the rate of Social Security claims for both sets of individuals affected by the law. The study found no statistically significant relationship between the Senior Citizens Freedom to Work Act of 2000 and earnings.
The quality of casual evidence presented in this report for Social Security claims and earnings outcomes is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000, but other factors might also have contributed. The quality of causal evidence presented in this report on employment is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000; other factors are likely to have contributed.","Senior Citizens Freedom to Work Act of 2000","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker",,"United States",2006,,"Employment and Training Review Protocol"
"Enforcement of civil rights law in private workplaces: The effects of compliance reviews and lawsuits over time","Kalev, A., & Dobbin, F. (2006). Enforcement of civil rights law in private workplaces: The effects of compliance reviews and lawsuits over time. Law & Social Equity, 31(4), 855-903.","Employer Compliance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:


The study’s objective was to examine the effectiveness of compliance reviews by the Office of Federal Contract Compliance Programs (OFCCP) and lawsuits initiated by individuals or the Equal Employment Opportunity Commission (EEOC) in reducing discrimination against women and African Americans in management positions.
The authors applied a fixed-effects regression model to workforce data in the EEO-1 Employer Information Reports. These data were supplemented by a survey of compliance enforcement activities administered to a random sample of establishments drawn from the EEO-1 reports.
The study found that compliance reviews begun in the 1970s increased an establishment’s share of women and African Americans employed as managers, not only in the 1970s but also through the 1980s and 1990s.
The quality of causal evidence presented in this report is low because the study did not adequately control for existing differences between the study groups. This means we are not confident that the estimated effects are attributable to OFCCP compliance reviews or to lawsuits. Other factors are likely to have contributed.","the Civil Rights Act of 1964 and Affirmative Action Executive Orders","Civil Rights Act of 1964 Equal Employment Opportunity Act of 1972 Executive Order 11246 (E.O. 11246)","Female, Black or African American",,"United States",2006,,"Employer Compliance Review Protocol"
"A comparison of two Welfare-to-Work strategies among African American women in Atlanta, Georgia. (Unpublished doctoral dissertation).","Wilson-Brewster, V.L. (2006). A comparison of two Welfare-to-Work strategies among African American women in Atlanta, Georgia. (Unpublished doctoral dissertation). Capella University, Minneapolis, Minnesota.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the effectiveness of two welfare-to-work programs—the Labor Force Attachment (LFA) program and the Human Capital Development (HCD) program—on the employment and earnings of African American women 25, 35, and 45 years old in Atlanta.
The author analyzed a subsample of participants in the National Evaluation of Welfare-to-Work Strategies (NEWWS) study. The study was a randomized controlled trial that assigned individuals within a geographic location to one of three groups: LFA, HCD, or a control group.
The study found that women in the 35-year-old group who participated in LFA or HCD had higher earnings and needed less time to find a job than those in the other age cohorts.
The quality of causal evidence presented in this report is low because the study was a randomized controlled trial with unknown attrition and the authors did not account for preexisting differences between the groups being compared in their analysis. This means we are not confident that the estimated effects are attributable to LFA or HCD programs; other factors are likely to have contributed.","Labor Force Attachment (LFA) and Human Capital Development (HCD)","Other employment and reemployment Other training and education","Female, Black or African American, Low income, Parent",,"United States, Urban",2006,,"Employment and Training Review Protocol"
"The labor market effects of sex and race discrimination laws.","Neumark, D., & Stock, W. (2006). The labor market effects of sex and race discrimination laws. Economic Inquiry, 44(3), 385-419.","Employer Compliance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Unfavorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to examine the effect of state-level sex and race discrimination laws passed before 1960 on employment and earnings outcomes for women and African Americans.
The authors used a triple difference estimator including state, year, and subgroup variables. The primary data source was U.S. Census data from 1940 to 1960.
The study found that sex-related equal pay laws had negative, statistically significant effects on women’s employment relative to men and a growing positive effect on women’s earnings relative to men over time. Racial discrimination laws had no significant immediate effects on employment or earnings for African American men compared with white men in the short term, but there were positive effects after the laws were in effect for a number of years.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects were attributable to the state sex and race discrimination laws; other factors are likely to have contributed.","State Sex and Race Discrimination Laws","Affirmative action","Female, Male, Black or African American",,"United States",2006,,"Employer Compliance Review Protocol"
"Teaching strategies designed to change the undergraduate experience for college women learning chemistry","Khan, S. (2005). Teaching strategies designed to change the undergraduate experience for college women learning chemistry. Journal of Women and Minorities in Science and Engineering 11, 365-387.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts
      
    
  
              


      
            Education and skills gains","Summary:


The study’s objective was to examine the impact of three teaching innovations—a contract, confidence-building exercises, and encouragement to apply for science internships—on performance in an Organic Chemistry II course at an all-female college.
The author compared the final exam scores, final grades, and internship acceptance rates of students enrolled in the course that included these interventions to outcomes for students who had taken the course without these interventions in the three prior years. All courses were taught by the same professor and used the same syllabus.
The study identified no statistically significant relationships between the three teaching innovations and students’ outcomes.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the teaching innovations; other factors are likely to have contributed.","Innovative Teaching for Women in STEM","Science, Technology, Engineering, and Math (STEM) programs","Youth, Female",,"United States",2005,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The effects of welfare-to-work program activities on labor market outcomes","Dyke, A., Heinrich, C., Mueser, P., & Troske, K. (2005). The effects of welfare-to-work program activities on labor market outcomes. Institute for Research on Poverty, 1295(5). [North Carolina]","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine the impact of North Carolina’s welfare-to-work program training activities on the quarterly earnings of women ages 18 to 65 years in single-parent households who were cash recipients of Temporary Assistance for Needy Families (TANF).
Using data from the state’s administrative records and unemployment insurance program, the authors compared the quarterly earnings of women who took part in the training activities with those who enrolled in the welfare-to-work program but never took part in any of the training.
The study found that those taking part in any of the training activities experienced an initial (quarter 1) reduction in their quarterly earnings, compared with those who did not take part in the training activities. However, those who took part in the assessment or intensive training activities experienced an increase in both their cumulative (based on 16 quarters of follow-up) and average (based on quarters 11 to 15 of follow-up) quarterly earnings, compared with those who did not take part in any training activities.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the welfare-to-work program, but other factors might also have contributed.","the Welfare-to-Work Program","Job search assistance and supportive services","Female, Low income, Parent",,"United States",2005,http://repec.iza.org/dp1520.pdf,"Employment and Training Review Protocol"
"The effects of welfare-to-work program activities on labor marker outcomes","Dyke, A., Heinrich, C., Mueser, P., & Troske, K. (2005). The effects of welfare-to-work program activities on labor marker outcomes. Institute for Research on Poverty, 1295(5). [Missouri]","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts
      
    
  
              


      
            Earnings and wages","Summary:

The study’s objective was to examine the impact of Missouri’s welfare-to-work program training activities on the quarterly earnings of women ages 18 to 65 years in single-parent households who were cash recipients of Temporary Assistance for Needy Families (TANF).
Using data from the state’s administrative records and unemployment insurance program, the authors compared the quarterly earnings of women who took part in the training activities with those who enrolled in the welfare-to-work program but never took part in any of the training.
The study found that those taking part in the training activities experienced an initial (quarter 1) reduction in their quarterly earnings, compared with those who did not take part in the training activities. However, both cumulative (based on all 16 quarters of follow-up) and average (based on quarters 11 to 15) quarterly earnings increased for those who took part in the training activities compared with those who did not.
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the welfare-to-work program training activities, but other factors might also have contributed.","the Welfare-to-Work Program","Job search assistance and supportive services","Female, Low income, Parent",,"United States",2005,http://repec.iza.org/dp1520.pdf,"Employment and Training Review Protocol"
"The effect of the SSI program on labor supply: Improved evidence from Social Security administrative files","Neumark, D., & Powers, E. T. (2005). The effect of the SSI program on labor supply: Improved evidence from Social Security administrative files. Social Security Bulletin, 65(3), 45-60.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study examined the relationship between generous state Supplemental Security Income (SSI) benefits and the employment of older worker nearing SSI eligibility age.
The authors used a regression model and data from the Survey of Income and Program Participants (SIPP) and Social Security Administration (SSA) to estimate the effect.
The study found that older males who were likely SSI participants in generous states worked significantly fewer hours and had lower employment rates compared with those living in less generous states. The authors, using the SSA administrative records, also found that likely older participants (ages 60 to 64) worked significantly fewer hours than younger ones.
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to generosity of state SSI benefits; other factors are likely to have contributed.","state Supplemental Security Income (SSI) benefits","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2005,https://www.ssa.gov/policy/docs/ssb/v65n3/v65n3p45.html,"Employment and Training Review Protocol"
"4 schools for WIE","Erkut, S., Marx, F., & Wellesley College Center for Research on Women. (2005). 4 schools for WIE. Evaluation report. Wellesley, MA: Wellesley Centers for Women.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Attitudes-Low-Mixed impacts
      
    
  
              


      
            Attitudes","Summary:


The study’s objective was to determine the effect of the 4 Schools for Women in Engineering (WIE) strategy on 8th-grade students’ attitudes toward STEM careers. The strategy involved forming teams of mostly female engineering professors, students, and practitioners to train middle school teachers on engineering and gender-inclusive teaching methods and to serve as in-class role models for students.
The authors administered a study-designed survey at the beginning and end of the school year to a single cohort of students to measure changes in outcomes following implementation of the intervention. The authors estimated program impacts using regression models and paired t-tests.
The study found that students’ attitudes toward science improved after the intervention, but attitudes toward engineering worsened. There was no change in students’ attitudes toward math.
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the 4 Schools for WIE strategy; other factors are likely to have contributed.","4 Schools for Women in Engineering","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2005,http://files.eric.ed.gov/fulltext/ED500754.pdf,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"More than robots: An evaluation of the FIRST robotics competition participant and institutional impacts","Melchior, A., Cohen, F., Cutter, T., & Leavitt, T. (2005). More than robots: An evaluation of the FIRST robotics competition participant and institutional impacts. Waltham, MA: Brandeis University Center for Youth and Communities Heller School for Social Policy and Management.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to examine the impact of the FIRST Robotics Competition (FRC) on participants’ education and career outcomes. The FRC was a six-week competition in which teams of high school male and female students (with the help of mentors) used a set of standard parts to build a robot. Students showcased these robots by competing against teams from other high schools.
The authors used propensity-score matching to create treatment and comparison groups and conducted an analysis of retrospective survey data to measure education and career outcomes of FRC participants who graduated from the program from 1999 to 2003. They administered a researcher-designed survey to students in the treatment group in 2004 and collected data from a national survey—the Beginning Postsecondary Student Survey (BPSS)—to measure outcomes for the comparison group, who began college in 1995-1996.
The study found that a higher percentage of female FRC participants majored in engineering compared with females in the comparison group.
The quality of causal evidence presented in this report is low because the authors did not demonstrate equivalency of the treatment and comparison groups on all required measures. This means we are not confident that the estimated effects are attributable to the FRC; other factors are likely to have contributed.","FIRST Robotics Competition","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female, Male",,"United States",2005,http://www.techfire225.com/uploads/6/3/7/1/6371896/first_study.pdf,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Persistent Problems Demand Consistent Solutions: Evaluating Policies to Mitigate Occupational Segregation by Gender","Mastracci, S.H. (2005). Persistent Problems Demand Consistent Solutions: Evaluating Policies to Mitigate Occupational Segregation by Gender. Review of Radical Political Economics, 37(1), 23-38.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts
      
    
  
              


      
            Employment","Summary:

The study’s objective was to determine the effect of two competitive grants that funded activities to support women in nontraditional occupations on women’s probability of employment in those occupations. Nontraditional occupations are those in which women represent fewer than one-quarter of all employed people in the occupation (for example, dentists or construction workers).
The author estimated the grants’ impact on female employment in nontraditional occupations through a differences-in-differences model using Current Population Survey Merged Outgoing Rotation Groups data from 1990 to 1999.
The study found that the grants were associated with increases in women’s chances of obtaining employment in a nontraditional occupation.
The quality of causal evidence presented in this report is low because the author did not control or account for pretreatment trends in the outcome variables of interest. This means we are not confident that the estimated effects are attributable to the grants; other factors are likely to have contributed.","the NEW and WANTO Grant Programs","Other disparities or discrimination in employment and earnings Science, Technology, Engineering, and Math (STEM) programs",Female,,"United States",2005,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Do scientists help people? Beliefs about scientists and the influence of pro-social context on girls’ attitudes toward physics","Yanowitz, K. (2004). Do scientists help people? Beliefs about scientists and the influence of pro-social context on girls’ attitudes toward physics. Journal of Women and Minorities in Science and Engineering, 10(4), 393-399.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-No impacts
      
    
  
              


      
            Attitudes","Summary:

The study’s objective was to examine the impact of reading a passage featuring a pro-social context on careers in physics on female students’ interest in science.
The authors randomly assigned 88 5th- and 6th-grade girls to four conditions. In the treatment condition, participants read a story describing a female physicist who entered that profession to benefit society. Participants assigned to the control conditions read different stories. After reading these stories, the participants responded to a short survey asking how much they liked the story and how interested they were in becoming physicists.
The study found no relationship between the treatment story and participants’ interest in becoming physicists.
The quality of causal evidence presented in this study is high because it was based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects would be attributable to the scientists’ stories and not to other factors. However, the study did not find statistically significant effects.",,"Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2004,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Student enrollment in high school AP sciences and calculus: how does it correlate with STEM careers?","Robinson, M. (2003). Student enrollment in high school AP sciences and calculus: how does it correlate with STEM careers? Bulletin of Science, Technology & Society, 23, 265–273.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

This study’s objective was to examine differences in the intended choice of college science, technology, engineering, and mathematics (STEM) majors between females and males enrolled in high school advanced placement (AP) science and calculus courses.
The study used survey data from eight high schools to examine the percentage of AP students who chose a STEM field as their intended college major. The authors compared outcomes for minority and nonminority males and females.
The study found that among students taking AP science or calculus classes, males were generally more likely than females to plan to choose a college major in engineering but less likely to plan to choose a college major in medicine.
The quality of causal evidence presented in this study is low. This means we are not confident that the estimated effects are attributable to AP courses. Other factors are likely to have contributed.","the Advanced Placement (AP) Program","Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female, Male",,"United States",2003,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"Alleviating women’s mathematics stereotype threat through salience of group achievements","McIntyre, R., Paulson, R., & Lord, C. (2003). Alleviating women’s mathematics stereotype threat through salience of group achievements. Journal of Experimental Social Psychology, 39, 83-90.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Education and skills gains-Mod/high-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

The study’s objective was to examine the impact of alleviating women’s stereotype threat—that is, the idea that men outperform women in mathematics—on women’s subsequent performance on a difficult mathematics test.
The study contained two experiments in which participants were randomly assigned to either a treatment condition, which received an intervention to alleviate women’s stereotype threat, or a control condition, which did not receive the intervention. The primary outcome of interest was an adjusted score for a study-administered mathematics test consisting of difficult questions from the Graduate Record Examination (GRE), an entry test for graduate school.
The study found that women in the treatment group scored 2.0 to 2.5 points higher on the math test than women in the control condition. These results were statistically significant.
The quality of causal evidence presented in this study is high because it was a well-conducted randomized controlled trial. This means that we are confident that the estimated effects are attributable to the intervention, and not to other factors.",,"Preventing discrimination Science, Technology, Engineering, and Math (STEM) programs Youth programs","Youth, Female",,"United States",2003,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol"
"The impact of Equal Employment Opportunity Commission enforcement on the wages of African American and white women, 1988–1996","Wilhelm, S. (2002). The impact of Equal Employment Opportunity Commission enforcement on the wages of African American and white women, 1988–1996. The Review of Black Political Economy, 30(2), 25-51.","Employer Compliance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages","Summary:


The study’s objective was to examine the impact of employment discrimination charges filed with the Equal Employment Opportunity Commission (EEOC) on the wages of African American and white women.
The author used regression analyses to compare the effects of sex- and race-based discrimination charges within an industry in a given year on the wages of African American and white women from 1988 to 1996. Data came from the Current Population Survey and the EEOC National Database.
The study found that increases in employment discrimination charges based on sex were positively correlated with increases in wages for both African American and white women. 
The quality of causal evidence provided in this study is low because the study did not adequately control for existing differences between the study groups. This means that we are not confident that the estimated effects are attributable to EEOC enforcement. Other factors are likely to have contributed.","Equal Employment Opportunity Commission Enforcement","Equal Employment Opportunity Act of 1972","Female, Black or African American, White",,"United States",2002,,"Employer Compliance Review Protocol"
"The labor market impact of state-level anti-discrimination laws","Collins, W. (2001). The labor market impact of state-level anti-discrimination laws. National Bureau of Economic Research working paper no. 8310. Cambridge, MA: NBER.","Employer Compliance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts
      
    
  
              


      
            Earnings and wages
      


  
      
            Employment-Low-Mixed impacts
      
    
  
              


      
            Employment","Summary:


This study’s objective was to assess the impact of state-level fair employment legislation, enacted in the 1940s through the 1960s, on labor market outcomes including income, unemployment, and labor force participation, by gender and race.
The study used a nonexperimental design, drawing on data from the 1940, 1950, and 1960 decennial U.S. Censuses for 19 non-southern states.
The study found that state-level fair employment laws adopted in the 1940s had larger positive effects on the relative income of African American workers than those adopted in the 1950s. In contrast, laws adopted in the 1950s led to larger increases in African American male unemployment but larger increases in African American women’s labor force participation. Only states that implemented the laws in the 1940s experienced a narrowed income gap between white and African American workers.
The quality of causal evidence presented in this report is low because the author did not demonstrate that the states that adopted fair employment legislation when they did were comparable to those that did not adopt or adopted at a later time. This means we are not confident that the estimated effects are attributable to state-level fair employment legislation. Other factors are likely to have contributed.","State-level Fair Employment Legislation","Affirmative action","Black or African American, Female, Male",,"United States",2001,http://www.nber.org/papers/w8310.pdf,"Employer Compliance Review Protocol"
"An evaluation of an urban community college Single Parent and Displaced Homemaker Program  (Doctoral dissertation)","Rice, L. M. (1999). An evaluation of an urban community college Single Parent and Displaced Homemaker Program  (Doctoral dissertation). Old Dominion University. DOI: 10.25777/tqh7-3c76","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts
      
    
  
              


      
            Education and skills gains","Summary:

	The study's objective was to examine the impact of the Single Parent and Displaced Homemaker Program on education outcomes by students at an urban community college.
	The study used a nonexperimental design to compare the outcomes of program participants to those on the waiting list. Using data from the community college’s Students Information System, the author conducted statistical tests to examine the differences between groups.
	The study found that participation in the Single Parent and Displaced Homemaker Program was significantly related to increased retention rates and number of credits taken.
	The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Single Parent and Displaced Homemaker Program; other factors are likely to have contributed.","the Single Parent and Displaced Homemaker Program","Employment and Training Services Training and Education Capacity building programs Community college education and other classroom training Other training and education","Adult, Other barriers, Female, Parent",,"Urban, United States",1999,https://digitalcommons.odu.edu/urbanservices_education_etds/50/,"Community College Review Protocol"