Title,Citation,Topic_area,Study_type,Study_evidence_rating,Outcome_effectiveness,Findings,Intervention_program,Topics,Target_population,Firm_characteristics,Geographic_setting,Original_publication_date,Original_publication_link,"Review Protocol" "Synthesis Report: Reemployment Synthesis",,Reemployment,,,,"Findings: The majority of reemployment interventions reduced weeks of benefit receipt and amount of benefits paid. The reemployment interventions had varying degrees of effectiveness. The Reemployment and Eligibility Assistance (REA) program boosted short-and long-term employment and earnings. Job search assistance (JSA) services had favorable impacts on all outcomes examined, but employment and earnings impacts varied over time. Reemployment bonuses appear to work in the short term, but their long-term effects are not known. Lighter-touch interventions, such as profiling and changing employer contact requirements,yield more limited benefits.",,"Job search assistance and supportive services Other employment and reemployment Unemployment Insurance",Unemployed,,"United States",,,"Reemployment Review Protocol" "Synthesis Report: Behavioral Finance Synthesis: Findings",,"Behavioral Finance: Retirement",,,,"Findings: People have relatively limited knowledge about saving for retirement and can be induced to save more when provided with additional information. Making retirement more salient, by having people think of themselves in retirement or providing a target retirement date, can increase intentions to save and alter investment choices. People can become overwhelmed by the number of investment options they face; when this occurs, they tend to use simple rules to make decisions.",,"Employer provided retirement benefits","Adult, Older worker",,"United States",,,"Behavioral Finance: Retirement Review Protocol" "Synthesis Report: Research Synthesis: Opportunities for Youth",,"Opportunities for Youth",,,,"Findings: Successful programs often involved a substantial time commitment from participating youth. Many successful programs involved a job placement component or job search assistance. Positive impacts tended to be realized in the short term and fade over time. More information is needed on the replicability of some programs.",,"Youth programs","Youth, Disconnected youth",,,,,"Opportunities for Youth Review Protocol" "Oregon Credentials, Acceleration and Support for Employment (CASE) evaluation report: Results, key issues and implications for policy, practice and systems","Watrus, B., & Fercho, H. (2015). Oregon Credentials, Acceleration and Support for Employment (CASE) evaluation report: Results, key issues and implications for policy, practice and systems. Retrieved from https://www.roguecc.edu/Programs/CareerPathways/pdf/CASE%20final%20evaluation%20report_Final.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Credentials, Acceleration and Support for Employment (CASE) program on education and employment outcomes. The authors used a nonexperimental design to compare the outcomes of students who were in the CASE program to a matched comparison group. The study found a positive relationship between participation in the CASE program and program completion and employment; however, the study did not include tests of statistical significance. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar or use sufficient. This means we are not confident that the estimated effects are attributable to the CASE program; other factors are likely to have contributed.","the Credentials, Acceleration and Support for Employment (CASE) Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Unemployed, Other",,"United States",2020,https://www.roguecc.edu/Programs/CareerPathways/pdf/CASE%20final%20evaluation%2…,"Community College Review Protocol" "Reassessing the labor market effects of California’s minimum nurse staffing regulations","Harless, D. (2019). Reassessing the labor market effects of California’s minimum nurse staffing regulations. Health Economics, 28, 1226–1231. https://doi.org/10.1002/hec.3924",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of California’s minimum staffing regulations for hospital nurses on wages and employment.  The author used a non-experimental comparison group analysis to compare the outcomes of registered nurses (RNs) and workers in similar occupations in California and 15 other states from 2001 to 2008. Data from the National Sample Survey of Registered Nurses provided information on employment. Wage data came from the Occupational Employment Statistics program and the American Community Survey.   The study found that the employment of RNs grew significantly faster in California than in other states following the regulations, and wages of RNs in California grew significantly faster relative to other occupations in other states following the regulations.  The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to California’s minimum staffing regulations for hospital nurses; other factors are likely to have contributed.  ",,"Compensation and Workplace Conditions Employment and Reemployment","Adult, Employed",,,2019,https://doi.org/10.1002/hec.3924,"Review Protocol" "Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report","Klerman, J. A., Saunders, C., Dastrup, E., Epstein, Z., Walton, D., Adam, T., & Barnow, B. S. (2019). Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report. Cambridge, MA: Abt Associates. [Comparison between multiple REA treatment group and single REA treatment group]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Reemployment and Eligibility Assessment (REA) program in four states (Indiana, New York, Washington, and Wisconsin) on people’s public benefit receipt, employment, and earnings. The authors investigated similar research questions for three other contrasts, the profiles of which are available using the study search. The authors used a randomized controlled trial to compare public benefit receipt outcomes among unemployment insurance (UI) claimants randomly assigned to states’ multiple REA treatment group as well as with the single REA treatment group. The authors drew on state administrative records from the UI benefit and case management systems for 28 weeks after random assignment. For this contrast, the study used data from New York and Washington. The study found that weeks of UI benefits were significantly lower for the multiple REA treatment group than the single REA treatment group. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors.","the Reemployment and Eligibility (REA) Program","RESEA Unemployment Insurance",Unemployed,,"United States",2019,https://www.dol.gov/sites/dolgov/files/OASP/evaluation/pdf/REA%20Impact%20Study…,"Reemployment Review Protocol" "Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report","Klerman, J. A., Saunders, C., Dastrup, E., Epstein, Z., Walton, D., Adam, T., & Barnow, B. S. (2019). Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report. Cambridge, MA: Abt Associates. [Comparison between existing REA group and partial REA group]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Reemployment and Eligibility Assessment (REA) program in four states (Indiana, New York, Washington, and Wisconsin) on people’s public benefit receipt, employment, and earnings. The authors investigated similar research questions for three other contrasts, the profiles of which are available using the study search. The authors used a randomized controlled trial to compare public benefit receipt and employment outcomes among unemployment insurance (UI) claimants randomly assigned to each state’s existing REA program as well as with a partial REA condition. The authors drew on state administrative records from the UI benefit and case management systems for 28 weeks after random assignment and National Directory of New Hires data for two years after random assignment. The study found that the existing REA treatment group received significantly fewer weeks of UI benefits and worked significantly more quarters than the partial REA treatment group. These findings were consistent in some of the individual states. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors.","the Reemployment and Eligibility (REA) Program","RESEA Unemployment Insurance",Unemployed,,"United States",2019,https://www.dol.gov/sites/dolgov/files/OASP/evaluation/pdf/REA%20Impact%20Study…,"Reemployment Review Protocol" "Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report","Klerman, J. A., Saunders, C., Dastrup, E., Epstein, Z., Walton, D., Adam, T., & Barnow, B. S. (2019). Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report. Cambridge, MA: Abt Associates. [Comparison between partial REA treatment group and control group]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Reemployment and Eligibility Assessment (REA) program in four states (Indiana, New York, Washington, and Wisconsin) on people’s public benefit receipt, employment, and earnings. The authors investigated similar research questions for three other contrasts, the profiles of which are available using the study search. The authors used a randomized controlled trial to compare public benefit receipt and employment outcomes among unemployment insurance (UI) claimants randomly assigned to a partial REA treatment group or a control group. The authors drew on state administrative records from the UI benefit and case management systems for 28 weeks after random assignment and National Directory of New Hires data for two years after random assignment. The study found that, across the four states, the partial REA treatment group received significantly fewer weeks of UI benefits. The study found no statistically significant differences between the groups in the number of quarters employed or longest job tenure one or two years after random assignment. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors.","the Reemployment and Eligibility (REA) Program","RESEA Unemployment Insurance",Unemployed,,"United States",2019,https://www.dol.gov/sites/dolgov/files/OASP/evaluation/pdf/REA%20Impact%20Study…,"Reemployment Review Protocol" "The effect of wage subsidies on piece rate workers: Evidence from the penny per pound program in Florida","Ku, H. (2019). The effect of wage subsidies on piece rate workers: Evidence from the penny per pound program in Florida. Journal of Development Economics, 139, 122–134. doi:10.1016/j.jdeveco.2019.03.005",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the impact of Penny Per Pound (PPP) on daily hours worked.  The author used a statistical model to compare the outcomes of workers at a Florida farm who were and were not affected by the program, using data from the farm’s payroll records and the National Oceanic and Atmospheric Administration’s National Climatic Data Center Global Surface Summary of Day Data.  The study found that the PPP was associated with a 22 percent decrease in daily hours worked per field.    The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to PPP; other factors are likely to have contributed.","Penny Per Pound","Compensation and Workplace Conditions","Adult, Employed",,,2019,https://www.sciencedirect.com/science/article/pii/S030438781830498X,"Review Protocol" "Information technology career preparation for rural areas.","Hahs-Vaughn, D. L., Swan, B., & Clark, M. H. (2019). Information technology career preparation for rural areas. Community College Journal of Research and Practice, 1-13.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Florida Information Technology Careers for Rural Areas (XCEL-IT) program on education, employment, and earnings outcomes. The authors used a nonexperimental design to compare the differences in outcomes between XCEL-IT program participants and a comparison group. The study found that XCEL-IT program participation was significantly related to program completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the XCEL-IT program; other factors are likely to have contributed.","the Florida Information Technology Careers for Rural Areas (XCEL-IT)","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Unemployed, Veteran or military",,"Rural, United States",2019,,"Community College Review Protocol" "Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report","Klerman, J. A., Saunders, C., Dastrup, E., Epstein, Z., Walton, D., Adam, T., & Barnow, B. S. (2019). Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report. Cambridge, MA: Abt Associates. [Comparison between existing REA services group and control group]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Reemployment and Eligibility Assessment (REA) program in four states on people’s public benefit receipt, employment, and earnings. The authors investigated similar research questions for three other contrasts, the profiles of which are available using the study search. The authors used a randomized controlled trial to compare the public benefit, earnings, and employment outcomes among unemployment insurance (UI) claimants randomly assigned to each state’s existing REA program or a control group. The authors drew on state administrative records from the UI benefit and case management systems for 28 weeks after random assignment and National Directory of New Hires data for two years after random assignment. The study found that, across the states, the existing REA treatment group received significantly fewer weeks and dollars of UI benefits than the control group did, and the existing REA treatment group had higher employment, longer job tenure, and higher earnings than the control group in the first or second year after random assignment. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors.","the Reemployment and Eligibility (REA) Program","RESEA Unemployment Insurance",Unemployed,,"United States",2019,https://www.dol.gov/sites/dolgov/files/OASP/evaluation/pdf/REA%20Impact%20Study…,"Reemployment Review Protocol" "Third-party evaluation of MoSTEMWINs: Implementation, outcomes, and impact","Cosgrove, J., & Cosgrove, M. (2018). Third-party evaluation of MoSTEMWINs: Implementation, outcomes, and impact. St. Louis, MO: Cosgrove & Associates, LLC.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Missouri STEM Workforce Innovation Networks (MoSTEMWINs) program on education and employment outcomes. The authors used a nonexperimental design to compare education and employment outcomes of MoSTEMWINs participants to a historical comparison group. The study found that MoSTEMWINs participants were significantly more likely to complete the program and attain employment than the comparison group. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MoSTEMWINs program; other factors are likely to have contributed.",MoSTEMWINs,"Basic skills Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Unemployed, Veteran or military",,"United States",2018,http://www.skillscommons.org/bitstream/handle/taaccct/10845/MoSTEMWINs%20Mid-Po…,"Community College Review Protocol" "Minnesota advanced manufacturing project: Final external evaluation report","Bucci, P., Evans, W.D., Friesenborg, L., Fox, D.A., Bishop, K., Balraj, N., & Jang, B. (2018). Minnesota advanced manufacturing project: Final external evaluation report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/18521/MnAMP%20Final%20Evaluation%20Report.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Mixed impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Minnesota Advanced Manufacturing Partnership (MnAMP) program on education outcomes. The authors used a nonexperimental design to compare education outcomes of MnAMP participants to a matched comparison group using institutional data. The study found that MnAMP program participation was significantly associated with earning fewer credits and fewer credentials relative to the comparison group. However, among students who attained credentials, participating in the MnAMP program was significantly associated with completing a one-year certificate in less time than comparison group students. The quality of causal evidence presented in this study is low because the authors used a comparison group from previous enrollment years, presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MnAMP program; other factors are likely to have contributed.","Minnesota Advanced Manufacturing Partnership (MnAMP) Program","Capacity building programs Community college education and other classroom training","Adult, Unemployed, Veteran or military",,"United States",2018,https://www.skillscommons.org/bitstream/handle/taaccct/18521/MnAMP%20Final%20Ev…,"Community College Review Protocol" "Preparing Alaskans for mining careers through short, industry-informed training programs: University of Alaska round IV TAACCCT grant","Hanson, H., Stevens, D., Vazquez, M., & Roberts, B. (2018). Preparing Alaskans for mining careers through short, industry-informed training programs: University of Alaska round IV TAACCCT grant. Portland, OR: Education Northwest.","Community College",,,"Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the University of Alaska's Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant program on the employment and earnings of community college students. The study used a nonexperimental design to compare the outcomes of students in the TAACCCT-enhanced Surface Mining and Underground Mining programs to similar individuals who did not attend community college and instead entered the labor market in mining occupations. The study found that the TAACCCT-enhanced program participation was significantly associated with a higher likelihood of employment retention and increased quarterly wages. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the University of Alaska's TAACCCT grant program; other factors are likely to have contributed.","the University of Alaska's TAACCCT Grant Program","Employer partnerships Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, American Indian or Alaska Native, Veteran or military",,"United States",2018,http://www.skillscommons.org/bitstream/handle/taaccct/18578/AK_TAACCCT_Report_0…,"Community College Review Protocol" "Long-term effects of job-search assistance: Experimental evidence using administrative tax data","Manoli, D. S., Michaelides, M., & Patel, A. (2018). Long-term effects of job-search assistance: Experimental evidence using administrative tax data. (Report no. w24422). Cambridge, MA: National Bureau of Economic Research.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study examined the impact of the Nevada Reemployment and Eligibility Assessment (REA) program on long-term employment, earnings, and Unemployment Insurance (UI) benefit receipt. This study was a randomized controlled trial that took place in the state of Nevada and involved new UI claimants from July to December 2009. The authors used program administrative and Internal Revenue Service (IRS) tax return data to compare outcomes of participants with outcomes of the control group for six years following the intervention. The study found that participants of the Nevada REA program experienced greater increases in employment and earnings relative to the control group for each of the six years following the intervention. Additionally, the study found that participants of the Nevada REA program experienced greater decreases in UI benefit receipt relative to the control group and lower benefit amounts in the first year after program entry. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Nevada REA program, and not to other factors. See more CLEAR profiles related to The Reemployment and Eligibility Assessment.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"Urban, United States",2018,http://www.daymanoli.com/wp-content/uploads/2018/03/Manoli_Michaelides_Patel-nv…,"Reemployment Review Protocol" "STEM education for workforce development through online contextualized training","Mohammadi, A. (2018). STEM education for workforce development through online contextualized training. (Unpublished doctoral dissertation). University of Nebraska, Lincoln, Nebraska.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of contextualized teaching and learning (CTL) on education outcomes. Using institutional data, the author conducted a nonexperimental design to compare education outcomes of CTL participants to non-CTL participants. The study found no statistically significant relationships between participation in the contextualized teaching and learning training and credit hours completed, length of program enrollment, or credentials attained. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention and did not include sufficient controls. This means we are not confident that the estimated effects are attributable to CTL; other factors are likely to have contributed.","the Contextualized Teaching and Learning (CTL) Training","Basic skills Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Other barriers, Dislocated or displaced worker",,"United States",2018,https://search.proquest.com/openview/2185911e68616901d0e3cf2e958e11d6/1?cbl=187…,"Community College Review Protocol" "KCTCS Enhancing Programs for IT Certification (EPIC): Independent evaluation contracted by the Kentucky Community and Technical College System","Hughes, K., Belfield, C., Ran, F., & Jenkins, D. (2018). KCTCS Enhancing Programs for IT Certification (EPIC): Independent evaluation contracted by the Kentucky Community and Technical College System. New York, NY: Community College Research Center, Teacher's College, Columbia University.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Education and skills gains-Mod/high-Favorable impacts Education and skills gains","Summary: The study’s objective was to assess the impact of the Enhancing Programs for IT Certification’s (EPIC) Learn on Demand (LoD) courses on community college students’ education outcomes. This summary focuses on the comparison between students who enrolled in EPIC LoD courses and students who enrolled in the traditional format courses. The authors used a nonexperimental design to compare the outcomes of students enrolled in EPIC LoD courses with those enrolled in traditional formats of courses where an EPIC LoD version was available. The study found that EPIC LoD course participation was significantly associated with increased course completion and certificate completion. The quality of causal evidence presented in this report is moderate for the course completion outcome because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the EPIC LoD courses, but other factors might also have contributed. For the certificate completion outcome, the quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the EPIC LoD courses; other factors are likely to have contributed.","the Enhancing Programs for IT Certification (EPIC) Project","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2018,https://ccrc.tc.columbia.edu/media/k2/attachments/CCRC_KCTCS_EPIC_Report_09-26-…,"Community College Review Protocol" "SUN PATH comparison group study","Dauphinee, T., & Bishwakarma, R. (2018). SUN PATH comparison group study. Albuquerque, NM: University of New Mexico, Career to Cradle Policy Institute.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Skill‐UP Network Pathways Acceleration in Technology and Healthcare (SUN PATH) program on education, earnings, and employment outcomes. The authors used a nonexperimental design to compare the outcomes of students who were in the SUN PATH program to a matched comparison group of students. The study found that SUN PATH students had higher rates of program completion, certificates/degree attainment, program retention, employment, and earnings than students in the comparison group. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the SUN PATH program, but other factors might also have contributed.","the Skill‐UP Network Pathways Acceleration in Technology and Healthcare (SUN PATH) Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Other, Low-skilled, Veteran or military",,"United States",2018,https://www.skillscommons.org/bitstream/handle/taaccct/18108/SUN%20PATH%20Compa…,"Community College Review Protocol" "Trade Adjustment Assistance Community College Career Training Grant (TAACCCT): Final evaluation report","The Greater Cincinnati Supply Chain Career Development Center (SCCDC). (2018). Trade Adjustment Assistance Community College Career Training Grant (TAACCCT): Final evaluation report. Cincinnati, OH: Author.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of The Greater Cincinnati Supply Chain Career and Development Center’s (SCCDC) Supply Chain Management and Materials Handling & Logistics programs on education outcomes. The study used a nonexperimental design to compare the program completion of students enrolled in SCCDC programs to students in a matched comparison group. The study found that participation in the Supply Chain Management program, one of the SCCDC programs, was associated with a significant increase in the likelihood of program completion. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the SCCDC programs; other factors are likely to have contributed.","the SCCDC’s Supply Chain Management and Materials Handling & Logistics Technologies Programs","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2018,https://www.skillscommons.org/handle/taaccct/17943,"Community College Review Protocol" "Knowledge to work summative evaluation: Lord Fairfax Community College.","Voorhees, R. A. (2018). Knowledge to work summative evaluation: Lord Fairfax Community College. Voorhees, NJ: Voorhees Group LLC.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to assess the impact of the Knowledge to Work (K2W) program on education outcomes. The study used a non-experimental design to compare participants in the K2W program with participants from a comparison group. The study found no statistically significant relationships between K2W program participation and education outcomes. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the K2W program; other factors are likely to have contributed.","the Knowledge to Work (K2W) Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Low-skilled",,"United States",2018,http://www.skillscommons.org/bitstream/handle/taaccct/17589/K2W%20Summative%20R…,"Community College Review Protocol" "Johnson County Community College (JCCC) TAACCCT final evaluation report","York, V. (2018). Johnson County Community College (JCCC) TAACCCT final evaluation report. Office of Educational Innovation and Evaluation. Manhattan, KS: Kansas State University.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to evaluate the effects of Johnson County Community College’s (JCCC) Accelerated, Collaborative Technology Training Services (ACTTS) project on education outcomes. The author used a nonexperimental design to compare cohorts of students enrolled in ACTTS and comparison programs on five educational outcomes of interest. The study found that ACTTS students were significantly more likely than comparison group participants to complete their program of study, be retained in their program of study, complete credit hours, and earn credentials. The quality of causal evidence presented in this study is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient controls. This means we are not confident that the estimated effects are attributable to ACTTS; other factors are likely to have contributed.","JCCC’s Accelerated, Collaborative Technology Training Services (ACTTS) project","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2018,https://www.skillscommons.org/bitstream/handle/taaccct/17872/JCCC%20TAACCCT%20F…,"Community College Review Protocol" "KCTCS Enhancing Programs for IT Certification (EPIC): Independent evaluation contracted by the Kentucky Community and Technical College System","Hughes, K., Belfield, C., Ran, F., & Jenkins, D. (2018). KCTCS Enhancing Programs for IT Certification (EPIC): Independent evaluation contracted by the Kentucky Community and Technical College System. New York, NY: Community College Research Center, Teacher's College, Columbia University.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Education and skills gains-Mod/high-Favorable impacts Education and skills gains","Summary: The study’s objective was to assess the impact of the Enhancing Programs for IT Certification’s (EPIC) Learn on Demand (LoD) courses on community college students’ education outcomes. This summary focuses on the comparison between students who enrolled in EPIC LoD courses and students who enrolled in non-EPIC LoD courses. The authors used a nonexperimental design to compare the outcomes of students enrolled in EPIC LoD courses with those enrolled in non-EPIC LoD courses. The study found that EPIC LoD course participation was significantly associated with increased course completion and certificate completion. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the EPIC LoD courses, but other factors might also have contributed.","the Enhancing Programs for IT Certification (EPIC) Project","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2018,https://ccrc.tc.columbia.edu/media/k2/attachments/CCRC_KCTCS_EPIC_Report_09-26-…,"Community College Review Protocol" "Improving the labor market outcomes of U.S. veterans: The long-run effect of the Transition Assistance Program","Li, X. (2018). Improving the labor market outcomes of U.S. veterans: The long-run effect of the Transition Assistance Program. New York: Syracuse University.",Veterans,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact the Transitional Assistance Program (TAP) had on outcomes related to employment, earnings, and education. The study used statistical models to compare veterans who had been offered and received TAP to veterans who were not offered TAP. Data came from the veterans supplement of the Current Population Survey for survey years 1995 to 2010. The study found positive associations between participation in TAP and employment, earnings, and education. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to TAP; other factors are likely to have contributed to the findings.","the Transitional Assistance Program","Job search assistance and supportive services Veterans' reemployment Other training and education","Dislocated or displaced worker, Veteran or military",,"United States",2018,,"Employment and Training Review Protocol" "Southern Regional Technical College TAACCCT: Healthcare Career Works! (HCW) program third-party evaluation final report","WorkED Consulting. (2018). Southern Regional Technical College TAACCCT: Healthcare Career Works! (HCW) program third-party evaluation final report. Burke, VA: WorkED Consulting.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Healthcare Career Works! (HCW) program on education outcomes. The authors used a nonexperimental design to compare differences in education outcomes between HCW participants and a matched comparison group. The study found no statistically significant relationships between the HCW program and program completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the HCW program; other factors are likely to have contributed.","Healthcare Career Works! (HCW)","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Unemployed, Veteran or military",,"United States",2018,,"Community College Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from St. Louis Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Evaluation of the Community College Consortium for Bioscience Credentials (c3bc): Final report.","Alamprese, J. A., Costelloe, S., Price, C., & Zeidenberg, M. (2017). Evaluation of the Community College Consortium for Bioscience Credentials (c3bc): Final report. Cambridge, MA: Abt Associates.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Unfavorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Community College Consortium for Bioscience Credentials (c3bc) redesigned courses on course completion rates. The authors used a nonexperimental design to compare course completion outcomes of c3bc participants to a matched comparison group. The study found that enrollment in c3bc courses was significantly associated with lower course completion rates. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the c3bc redesigned courses; other factors are likely to have contributed.","the Community College Consortium for Bioscience Credentials (c3bc) Course Redesigns","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker",,"United States",2017,https://www.abtassociates.com/insights/publications/report/evaluation-of-the-co…,"Community College Review Protocol" "Saddleback College TAACCCT grant final evaluation report.","Pacific Research and Evaluation, LLC. (2017). Saddleback College TAACCCT grant final evaluation report. Portland, OR: Pacific Research and Evaluation, LLC.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of Saddleback College’s Fast Track to Success program on student education outcomes. The study used a nonexperimental design to compare the outcomes of students who were in the Medical Assistant (MA) and Registered Nurse (RN) tracks of the Fast Track to Success program to a comparison group. The study found that MA and RN students in Fast Track to Success programming were significantly more likely to earn certificates, degrees, and credentials in the MA and RN areas. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to the Fast Track to Success program; other factors are likely to have contributed.","the Fast Track to Success Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/13850/Saddleback%20Colle…,"Community College Review Protocol" "Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs [Comparison Between Full-WIA Services versus Core-and-Intensive Services].","Fortson, K., Rotz, D., Burkander, P., Mastri, A., Schochet, P., Rosenberg, L., McConnell, S., & D'Amico, R. (2017). Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs (No. 42e8b3550e40408f854b966d0229c3b5). Washington, DC: Mathematica Policy Research. [Comparison Between Full-WIA Services versus Core-and-Intensive Services]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 30 months. This summary focuses on the comparison between the group who received the full-WIA services versus the group who received core-and-intensive services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 30 months after study enrollment. The study found that full-WIA customers were significantly more likely than core-and-intensive customers to complete a training program. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2017,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report. (Mississippi River Transportation, Distribution & Logistics Consortium, 2017)","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Hinds Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Does protecting older workers from discrimination make it harder to get hired? Evidence from disability discrimination laws.","Neumark, D., Song, J., & Button, P. (2017). Does protecting older workers from discrimination make it harder to get hired? Evidence from disability discrimination laws. Research on Aging, 39(1), 29-63.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts Employment","Summary: The study examined the impact of the strength of state disability discrimination laws on hiring rates of older men. The study used statistical models and data from the Health and Retirement Study and the Survey of Income and Program Participation to estimate the impacts. The study found no relationship between the strength of disability discrimination laws and the hiring rate for older unemployed disabled and nondisabled men. The quality of casual evidence presented in this report is low because the study is a nonexperimental analysis that did not demonstrate that the groups being compared were similar and did not account for possible differences in the analysis. This means we are not confident that the estimated effects are attributable to state disability discrimination laws; other factors are likely to have contributed.","Disability discrimination laws","Health Older workers' programs","Adult, Older worker, Male",,"United States",2017,https://journals.sagepub.com/doi/full/10.1177/0164027516656142,"Employment and Training Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Southwest Tennessee Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Orthotics, Prosthetics, and Pedorthics (HOPE) careers consortium: Final evaluation report","Good, K., & Yeh-Ho, H. (2017). Orthotics, Prosthetics, and Pedorthics (HOPE) careers consortium: Final evaluation report. Denver, CO: McREL International.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Orthotics, Prosthetics & Pedorthics (HOPE) careers consortium program on education outcomes. The authors used a nonexperimental design to compare education outcomes of HOPE participants to a matched comparison group with institutional research data. The study found no significant associations between HOPE participation and program completion rates, completion of more than one certificate or degree, or furthering education status. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the HOPE program; other factors are likely to have contributed.","the Orthotics, Prosthetics & Pedorthics (HOPE) Careers Consortium Program","Capacity building programs Community college education and other classroom training","Adult, Other barriers, Dislocated or displaced worker, Unemployed, Veteran or military",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/13872/hope-consortium-tc…,"Community College Review Protocol" "Final evaluation report: Cincinnati State Technical and Community College: Greater Cincinnati Manufacturing Careers Accelerator (GCMCA)","Belville, J., Romano, G., Frisby, M., Chamberlin, M., & Strem, L. (2017). Final evaluation report: Cincinnati State Technical and Community College: Greater Cincinnati Manufacturing Careers Accelerator (GCMCA).","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Unfavorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program on education outcomes. The author used a nonexperimental design to compare the education persistence of students in the treatment group to students in a matched comparison group. The study found that students in the comparison group were significantly more likely to complete the program than students in the treatment group for both the Welding or Mechanical Engineering Technology Manufacturing and Design programs. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention.This means we are not confident that the estimated effects are attributable to the Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) program; other factors are likely to have contributed.    ","Greater Cincinnati Manufacturing Careers Accelerator (GCMCA) Program","Basic skills Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2017,,"Community College Review Protocol" "Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs [Comparison Between Full-WIA Services versus Core Services].","Fortson, K., Rotz, D., Burkander, P., Mastri, A., Schochet, P., Rosenberg, L., McConnell, S., & D'Amico, R. (2017). Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs (No. 42e8b3550e40408f854b966d0229c3b5). Washington, DC: Mathematica Policy Research. [Comparison Between Full-WIA Services versus Core Services]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 30 months. This summary focuses on the comparison between the group who received the full-WIA services versus the group who received core services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 30 months after study enrollment. The study found that compared to core customers, full-WIA customers reported a higher number of hours worked, an increased likelihood to receive a credential and complete a training program, and a decreased likelihood to receive public assistance. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2017,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report.","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Delgado Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report.","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Minnesota State College – Southeast Technical. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that there was no statistically significant relationship between enrollment in grant-funded programming and program completion. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Butler Community College TAACCCT final evaluation report","Kansas State University Office of Educational Innovation and Evaluation. (2017). Butler Community College TAACCCT final evaluation report. Manhattan, KS: Kansas State University.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Butler Community College Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant on education, earnings, and employment outcomes. The study used a nonexperimental design to compare the outcomes of students who were in the Butler TAACCCT Information Technology (IT) program to a comparison group. The study found that students in the Butler TAACCCT IT program were significantly more likely to complete their program, be retained in their program, and earn credentials than their counterparts in other selected Career and Technical Education (CTE) courses. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention or include sufficient controls. This means we are not confident that the estimated effects are attributable to the Butler TAACCCT IT program; other factors are likely to have contributed.","the Butler Community College TAACCCT Project","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Veteran or military",,"United States",2017,https://www.skillscommons.org/handle/taaccct/15664,"Community College Review Protocol" "The effect of the Affordable Care Act on labor transitions among older workers.","Camilleri, S. E. (2017). The effect of the Affordable Care Act on labor transitions among older workers. In Three papers on the Affordable Care Act (Doctoral dissertation). North Carolina State University, Raleigh, NC.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-No impacts Employment","Summary: The study examined the impact of the Affordable Care Act (ACA) on the employment outcomes of older workers. The author used data from the Current Population Survey for 2011–2016 and regression analyses to estimate impacts. The study found that the ACA was not associated with employment outcomes among older workers. The quality of causal evidence presented in this report is moderate because the study is based on a well-implemented nonexperimental design. This means we are somewhat confident that any estimated effects are attributable to the ACA, but other factors might also have contributed.","Affordable Care Act (ACA)","Older workers' programs Health insurance","Adult, Older worker",,"United States",2017,,"Employment and Training Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report.","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Arkansas State University Mid-South. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that there was no statistically significant relationship between enrollment in grant-funded programming and program completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Final evaluation report: Linn-Benton Community College","Kelley-Smith, A., Schoeph, K., Hamai, T., & Bishop, M. (2017). Final evaluation report: Linn-Benton Community College. Albany, OR: LB iLearn Campus.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to assess the impact of degree and certificate programming offered at Linn-Benton Community College’s (LBCC) iLearn campus compared to traditional programming offered at the community college on various educational, employment, and earnings outcomes. The study used a nonexperimental design to compare the outcomes of students enrolled in the Linn-Benton (LB) iLearn campus to students enrolled in traditional programming. The study found that program participation was significantly associated with increased rates of program completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the LB iLearn campus program; other factors are likely to have contributed.","the LB iLearn Campus Program","Capacity building programs Community college education and other classroom training","Adult, Other barriers, Dislocated or displaced worker, Veteran or military",,"United States",2017,https://www.skillscommons.org/handle/taaccct/15675,"Community College Review Protocol" "Evaluation of Accelerated Training or Illinois Manufacturing (ATIM): Impact report.","Betesh, H., Kim, H., Kogan, D., Lindy, R., Paprocki, A. (2017). Evaluation of Accelerated Training for Illinois Manufacturing (ATIM): Impact report. Retrieved from https://www.spra.com/wordpress2/wp-content/uploads/2017/05/ATIM-Impact-Report_Final.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to assess the impact of Accelerated Training for Illinois Manufacturing (ATIM) Program, a workforce training intervention, on education, employment, and earnings outcomes. The authors used a randomized controlled trial to assess the program’s impact and compared education, employment, and earnings outcomes between the treatment and control group. Data on participant characteristics and outcomes was gathered from grant data systems, the Illinois Workforce Development System, and Illinois State Unemployment Insurance. The study found that significantly higher percentages of people in the treatment group than the control group enrolled in education training, completed training, and earned a certificate. In addition, those who were in the treatment group earned a significantly higher number of certificates than participants in the control group, and had significantly higher wages in the second year after beginning study participation. The quality of causal evidence presented in this report is high for the education outcomes because it was based on a randomized controlled trial with low attrition. This means we are confident that the estimated effects are attributable to the ATIM program, and not to other factors. The quality of causal evidence is moderate for the employment and earnings outcomes because these outcomes had high attrition but the authors included sufficient controls in their analysis. This means we are somewhat confident that the estimated effects are attributable to the ATIM program, but other factors might also have contributed.","Accelerated Training for Illinois Manufacturing (ATIM) Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Low-skilled",,"United States",2017,https://www.spra.com/wordpress2/wp-content/uploads/2017/05/ATIM-Impact-Report_F…,"Community College Review Protocol" "Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs [Comparison between Core-and-Intensive Services versus Core Services].","Fortson, K., Rotz, D., Burkander, P., Mastri, A., Schochet, P., Rosenberg, L., McConnell, S., & D'Amico, R. (2017). Providing Public Workforce Services to Job Seekers: 30-Month Impact Findings on the WIA Adult and Dislocated Worker Programs (No. 42e8b3550e40408f854b966d0229c3b5). Washington, DC: Mathematica Policy Research. [Comparison between Core-and-Intensive Services versus Core Services]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 30 months. This summary focuses on the comparison between the group who received core-and-intensive services versus the group who received core services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 30 months after study enrollment. The study found that compared to core customers, core-and-intensive customers were significantly more likely to receive a credential through a training program and have higher earnings. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2017,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "STEM-Connect at the University of Vermont, College of Engineering and Mathematical Sciences: Program evaluation final report","Farrell, R., Harris, D., Meyers, H., & Ratmeyer, S. (2017). STEM-Connect at the University of Vermont, College of Engineering and Mathematical Sciences: Program evaluation final report. Swanton, VT: Core Research and Evaluation.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the effects of the University of Vermont’s (UVM) Science, Technology, Engineering and Math (STEM) Connect program on earnings and employment outcomes. The study used a nonexperimental design to compare the outcomes of students who were enrolled in the UVM STEM-Connect program to a matched comparison group of students who took a STEM-Connect course but were not enrolled in the STEM-Connect program. The study found no statistically significant relationships between STEM-Connect program participation and average quarterly wages or employment rates. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the UVM STEM-Connect program; other factors are likely to have contributed.","the UVM STEM-Connect Program","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Unemployed, Veteran or military",,"United States",2017,http://www.skillscommons.org/bitstream/handle/taaccct/13127/STEM-Connect%20Fina…,"Community College Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report.","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from Lewis & Clark Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from West Kentucky Community and Technical College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly higher for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "The impact of Medicare Part D on self-employment.","Moulton, J. G., Diebold, J. C., & Scott, J. C. (2017). The impact of Medicare Part D on self-employment. Research on Aging, 39(1), 64-85.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study examined the effect of the introduction of Medicare Part D on self-employment. The authors used a nonexperimental approach and data from the American Community Survey to estimate the impact. The authors found that self-employment significantly increased by 0.5 percentage points after the program began. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects on self-employment are attributable to the introduction of Medicare Part D, although other factors might also have contributed.","Medicare Part D","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2017,,"Employment and Training Review Protocol" "Trade Adjustment Assistance Community College Career Training grant final report.","Mississippi River Transportation, Distribution & Logistics Consortium. (2017). Trade Adjustment Assistance Community College Career Training grant final report. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Report_9_30_2017.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Unfavorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Mississippi River Transportation, Distribution and Logistics (MRTDL) Consortium’s grant-funded programs on education outcomes. This summary contains the findings from John Wood Community College. The authors used a nonexperimental design to compare the outcomes of students enrolled in programs enhanced through grant funding to those enrolled in a similar program at the same community college. The study found that the odds of program completion were significantly lower for participants in the grant-funded programs compared to participants in the comparison group. The quality of causal evidence presented in this report is low because the authors used a comparison group from different academic programs and also used a comparison group from previous enrollment years presenting a confounding factor. This means we are not confident that the estimated effects are attributable to the MRTDL Consortium’s grant-funded programs; other factors are likely to have contributed.","the Mississippi River Transportation, Distribution and Logistics (MRTDL) TAACCCT Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2017,https://www.skillscommons.org/bitstream/handle/taaccct/14261/MRTDL%20Final%20Re…,"Community College Review Protocol" "Final evaluation Golden Triangle Modern Manufacturing Project","Harpole, S. (2017). Final evaluation Golden Triangle Modern Manufacturing Project. SHH Consulting, LLC","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Golden Triangle Modern Manufacturing Project on education, earnings, and employment outcomes. The author used a nonexperimental design to compare outcomes of Golden Triangle Modern Manufacturing Project participants to a matched comparison group. The study found that program participation was significantly associated with increases in program retention, credential attainment, and earnings. The quality of causal evidence presented in this study is low because the author used a comparison group from previous enrollment years presenting a confounding factor. This means we are not confident that the estimated effects are attributable to Golden Triangle Modern Manufacturing Project; other factors are likely to have contributed.","the Golden Triangle Modern Manufacturing Project","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Unemployed, Low-skilled, Veteran or military",,"United States",2017,https://www.skillscommons.org/handle/taaccct/15669,"Community College Review Protocol" "The effect of Social Security information on the labor supply and savings of older Americans.","Armour, P., & Lovenheim, M. F. (2016). The effect of Social Security information on the labor supply and savings of older Americans. (Working paper no. 361). Ann Arbor, MI: University of Michigan, Michigan Retirement Research Center.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Mixed impacts Employment","Summary: The study examined the impact of the Social Security statements issued by the Social Security Administration (SSA) on older men’s annual hours worked. The authors used a nonexperimental design and data from Health and Retirement Study surveys and Social Security administrative records to estimate the impacts. The study found that receiving the first Social Security Statement is associated with a decrease in self-reported annual hours worked and receiving the second Social Security Statement is associated with an increase in self-reported annual hours worked. The quality of casual evidence presented in this report is moderate because that the study was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to Social Security statements, but other factors might also have contributed.","Social Security statements","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2016,,"Employment and Training Review Protocol" "The effects of youth employment: Evidence from New York City lotteries.","Gelber, A., Isen, A., & Kessler, J. B. (2016). The effects of youth employment: Evidence from New York City lotteries. The Quarterly Journal of Economics, 131(1), 423-460.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Education and skills gains-Mod/high-No impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Summer Youth Employment Program (SYEP) on the employment rate, earnings, and college enrollment of New York City youth 14 to 21 years old. The authors compared the employment rate, earnings, and college enrollment of the treatment group, youth who won a random lottery for the SYEP program and chose to participate in it, with those of the control group, youth who applied to the program but were not randomly selected to participate. The study used data from the Internal Revenue Service to measure outcomes over a five-year period. The study found that, compared with earnings of the control group, the treatment group had significantly higher earnings in the year of participation in SYEP, but significantly lower earnings in the following three years. The treatment group also had a significantly higher probability of being employed in any job during the year of participation in SYEP and the first follow-up year compared with the employment probability of the control group. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the SYEP program, and not to other factors.","Summer Youth Employment Program (SYEP)","Youth programs","Youth, Disconnected youth",,"United States",2016,,"Review Protocol" "Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs [Comparison betwwen Full-WIA Services versus Core Services].","McConnell, S., Fortson, K., Rotz, D., Schochet, P., Burkander, P., Rosenberg, L., Mastri, A., & D’Amico, R. (2016). Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs. Washington, DC: Mathematica Policy Research. [Comparison betwwen Full-WIA Services versus Core Services]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 15 months. This summary focuses on the comparison between the group who received the full-WIA services versus the group who received core services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 15 months after random assignment. The study found that full-WIA customers were significantly more likely than core customers to complete a training program and receive a training credential. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2016,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "Final evaluation report: Implementation and outcomes of Credentials to Careers","Skilton-Sylvester, P., Myran, S., Myran, G., Ross, S., & Williams, M. (2016). Final evaluation report: Implementation and outcomes of Credentials to Careers. Canadian Lakes, MI: Myran & Associates LLC.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the Credentials to Careers (C2C) program on earnings. The authors used a nonexperimental design to compare outcomes of C2C participants to a matched comparison group using data provided by the college. The study found that C2C program participation was significantly associated with a larger average percent increase in wages relative to the comparison group. The quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the C2C program; other factors are likely to have contributed.","the Credentials to Careers (C2C) Program","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Unemployed",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/15623/Final%20C2C%20Impl…,"Community College Review Protocol" "Evaluation of the Illinois Network for Advanced Manufacturing: Final Report","Westat. (2016). Evaluation of the Illinois Network for Advanced Manufacturing: Final Report. Rockville, MD: Westat & GEM Software Development, Inc.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Illinois Network for Advanced Manufacturing (INAM) program on education, earnings, and employment outcomes. The study used a nonexperimental design to compare the outcomes of students who were in the INAM program to a comparison group of students who took similar courses before the creation of INAM. Using data from college databases and the Illinois Department of Employment Security, the authors conducted statistical models to examine differences in outcomes between the groups. The study found that participation in the INAM program was significantly associated with improved rates of certificate or degree completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the INAM program; other factors are likely to have contributed.","the Illinois Network for Advanced Manufacturing (INAM)","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Unemployed, Other, Veteran or military",,"United States",2016,https://www.researchgate.net/publication/315664265_Evaluation_of_the_Illinois_N…,"Community College Review Protocol" "Third-party evaluation of implementation & impact of the Southwest Missouri Public Safety and Emergency Medical Initiative TAACCCT grant.","Cosgrove & Associates. (2016). Third-party evaluation of implementation & impact of the Southwest Missouri Public Safety and Emergency Medical Initiative TAACCCT grant. St. Louis, MO: Cosgrove & Associates.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of Crowder College’s enhanced Public Safety and Emergency Medical Initiative (PSP) program on education and employment outcomes. The study used a nonexperimental design to compare the outcomes of students who were in Crowder College’s enhanced PSP program to a comparison group of students who were not enrolled in the program. The study found that participation in the PSP program was significantly associated with higher program completion and employment rates than the comparison group. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Crowder College’s enhanced PSP program; other factors are likely to have contributed.","Crowder College Public Safety and Emergency Medical Initiative (PSP) Program","Capacity building programs Community college education and other classroom training","Adult, Other barriers, Unemployed, Low-skilled",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/9128/Crowder%20College_P…,"Community College Review Protocol" "Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs [Comparison between Core-and-Intensive Services versus Core Services].","McConnell, S., Fortson, K., Rotz, D., Schochet, P., Burkander, P., Rosenberg, L., Mastri, A., & D’Amico, R. (2016). Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs.  Washington, DC: Mathematica Policy Research. [Comparison between Core-and-Intensive Services versus Core Services]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 15 months. This summary focuses on the comparison between the group who received core-and-intensive services versus the group who received core services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 15 months after random assignment. The study found that core-and-intensive customers were significantly more likely than core customers to complete a training program, receive a training credential, and have higher earnings. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2016,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "Third-party evaluation services for the Trade Adjustment Assistance Community College and Career Training grant final report","Selzer, A. K., Sanchez, R., Michaelides, M., Shetty, S., & Bertane, C. (2016). Arizona Sun Corridor Get Into Energy Consortium (ASC-GIEC): Third-party evaluation services for the Trade Adjustment Assistance Community College and Career Training grant final report. Columbia, MD: IMPAQ International.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Arizona Sun Corridor Get Into Energy Consortium (ASC-GIEC) program on student education outcomes. The study used a nonexperimental design to compare the education outcomes of students who were in the ASC-GIEC program to a comparison group of students in other Career and Technical Education (CTE) programs. The study found that participation in the ASC-GIEC program was associated with higher program completion rates than the comparison program. However, the study did not include tests of statistical significance for this outcome. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the ASC-GIEC program; other factors are likely to have contributed.","the Arizona Sun Corridor Get Into Energy Consortium (ASC-GIEC) Program","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/15587/Estrella%20Mountai…,"Community College Review Protocol" "Final evaluation report: Summative evaluation TAACCCT round 2 accelerated, blended – online, 2 + 2 TAACCCT training project","Franco, E. O. (2016). Final evaluation report: Summative evaluation TAACCCT round 2 accelerated, blended – online, 2 + 2 TAACCCT training project. Cayey, PR: Colegio Universitario de San Juan.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Colegio Universitario de San Juan’s (CUSJ) modified electronics, engineering, and technology programs on education outcomes. The study used a nonexperimental design to compare education outcomes of CUSJ participants to a comparison group. Using CUSJ’s embedded course assessments, the author conducted statistical tests to examine differences between the groups. The study found that CUSJ participation was significantly associated with higher approval rates (defined as passing a course). The quality of causal evidence presented in this report is low because the author did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the CUSJ course modifications; other factors are likely to have contributed.","the Colegio Universitario de San Juan’s (CUSJ) Modified Courses","Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Unemployed",,"United States",2016,https://www.skillscommons.org/handle/taaccct/15668,"Community College Review Protocol" "Contra Costa Community College District Design it–Build it–Ship it (DBS) final evaluation report","Rayyes, N., Abe, Y., Sanchez, R., Lai, F., Akiya, K., Chan, V., & Jennings, E. D. (2016). Contra Costa Community College District Design it–Build it–Ship it (DBS) final evaluation report. Oakland, CA: Impaq International.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts Earnings and wages Education and skills gains-Mod/high-Unfavorable impacts Education and skills gains Employment-Mod/high-No impacts Employment","Summary: The study’s objective was to examine the impact of the Design it–Build it–Ship it (DBS) program on education, employment, and earnings outcomes. The authors used a nonexperimental design to compare education, earnings, and employment outcomes of DBS participants to a comparison group. The study found that DBS participation was significantly associated with lower rates of enrollment, fewer credentials/degrees earned, lower transfer rates to four-year colleges, and lower wages. The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the DBS program, but other factors might also have contributed.","the Design it–Build it–Ship it (DBS) Program","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Unemployed",,"United States",2016,https://www.impaqint.com/sites/default/files/files/IMPAQ%20FINAL%20DBS%20Evalua…,"Community College Review Protocol" "Effects of the unemployment insurance work test on long-term employment outcomes.","Lachowska, M., Meral, M., & Woodbury, S.A. (2016). Effects of the unemployment insurance work test on long-term employment outcomes. Labour Economics, 41, 246–265.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of more-stringent work search requirements for unemployment insurance (UI) claimants on earnings, employment, and UI benefit receipt outcomes over a nine-year follow-up period. This study extended the work of Johnson and Klepinger (1991), the CLEAR profile of which is available here. The CLEAR profile of a related study, Lachowska et al. (2015), is available here. The study examined data from a randomized controlled trial conducted in the 1980s in Washington State. UI claimants were randomly assigned to one of four groups, each with different work search requirements and/or verification of that work search. The authors used administrative data to compare the outcomes of the groups up to nine years after the initial UI claims. The study found that the groups with more-stringent work search requirements were more likely to be employed than the group with less-stringent requirements in the first year following their claims. They also received UI benefits payments for fewer weeks, exhausted UI benefits at a lower rate, and received fewer conditional payments in the year following their initial UI claims. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the more stringent work search requirements, and not to other factors.","More stringent work search requirements","RESEA Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2016,,"Reemployment Review Protocol" "Third party evaluation of MoManufacturingWINS: Implementation, outcomes, and impact","Cosgrove, J. J., Cosgrove, M. S., & Bragg, D. D. (2016). Third party evaluation of MoManufacturingWINs: Implementation, outcomes, and impact. St. Louis, MO: Cosgrove & Associates and Bragg & Associates.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of Missouri’s Manufacturing Workforce Innovation Networks(MMW) program on student education and employment outcomes. The study used a nonexperimental design to compare the outcomes of students who were in MMW to a comparison group of students in other manufacturing-technology related programs. The study found that MMW participation was significantly associated with higher retention and employment rates. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention and they did not include sufficient controls. This means we are not confident that the estimated effects are attributable to MMW; other factors are likely to have contributed.","Missouri’s Manufacturing Workforce Innovation Networks (MMW)","Basic skills Capacity building programs Community college education and other classroom training","Adult, Unemployed, Other, Low-skilled, Veteran or military",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/10116/ThirdPartyFinalEva…,"Community College Review Protocol" "Job insecurity, unemployment insurance and on-the-job search. Evidence from older American workers.","Gutierrez, I. A. (2016). Job insecurity, unemployment insurance and on-the-job search. Evidence from older American workers. Labour Economics, 41, 228-245. doi:10.1016/j.labeco.2016.05.011","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment","Summary: The study examined the impact of the potential wage replacement rate through Unemployment Insurance (UI) on the probability of transitioning to non-employment. The study used a nonexperimental design and data from the Health and Retirement Study (HRS) from 1996–2006 and 2010–2012 to identify workers who expected job loss during the baseline survey and who experienced job transitions by the follow-up survey two years later. The study found no statistically significant relationships between potential wage replacement rates and transitions to non-employment. The quality of causal evidence presented in this report is low because the analyses do not sufficiently account for other factors that might contribute to the estimated effects. This means we are not confident that the estimated effects are attributable to the potential wage replacement rate through UI; other factors are likely to have contributed.","Unemployment Insurance (UI)","Older workers' programs Unemployment Insurance","Adult, Older worker, Employed",,"United States",2016,,"Employment and Training Review Protocol" "Spousal labor market effects from government health insurance: Evidence from a veterans affairs expansion.","Boyle, M. A. & Lahey, J. N. (2016). Spousal labor market effects from government health insurance: Evidence from a veterans affairs expansion. Journal of Health Economics, Elsevier, 45(C), 63-76.","Older Workers, Veterans","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Mod/high-No impacts Employment","Summary: The study examined the impact of the expansion of health insurance for veterans through the U.S. Department of Veterans Affairs (VA) in the mid-1990s (as a result of the implementation of the Veteran’s Health Care Eligibility Reform Act) on the employment and earnings of non-covered spouses. The authors used a statistical model and data from the Census Bureau’s March Current Population Survey and the Health and Retirement Study to compare the employment and earnings of wives of male veterans with wives of non-veterans of the same ages before and after the expansion of health insurance for veterans through the VA. The study found no statistically significant associations between the expansion of health insurance for veterans and the employment of their wives and mixed evidence on associations between the expansion of health insurance for veterans and the earnings of their wives. The quality of causal evidence presented in this report is moderate for the employment outcomes because it was based on a well-implemented nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to the expansion of health insurance for veterans through the VA, but other factors might also have contributed. The quality of causal evidence presented in this report is low for earnings outcomes because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the expansion of health insurance for veterans through the VA; other factors are likely to have contributed.","Veteran Health Insurance Expansion","Health Older workers' programs Veterans' reemployment","Adult, Older worker, Veteran or military",,"United States",2016,,"Older Workers, Veterans Review Protocol" "Evaluation of the Alabama/Florida Technical Employment Network TAACCCT program","PTB & Associates. (2016). Evaluation of the Alabama/Florida Technical Employment Network TAACCCT program. Bethesda, MD: PTB & Associates.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Unfavorable impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Alabama-Florida Technical Employment Network (AF-TEN) on student education and employment outcomes. The study used a nonexperimental design to compare the education and employment outcomes of students who were in AF-TEN-enhanced welding programs to a matched comparison group of program students from prior years. The study found that participation in the AF-TEN programs was significantly associated with fewer completed credit hours. The quality of causal evidence presented in this report is low because the authors used a comparison group from previous enrollment years, which presents a confound. This means we are not confident that the estimated effects are attributable to the AF-TEN intervention; other factors are likely to have contributed.","the Alabama-Florida Technical Employment Network (AF-TEN)","Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/15590/George%20C.%20Wall…,"Community College Review Protocol" "Final evaluation of Southwest Virginia Community College’s PluggedInVA TAACCCT Grant initiative","Topper, A., Munn, W., Bor, E., Clery, S., & Smith, B. (2016). Final evaluation of Southwest Virginia Community College’s PluggedInVA TAACCCT Grant initiative. Retrieved from: https://www.skillscommons.org//handle/taaccct/15650","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-No impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the PluggedInVA (PIVA) program on education and employment outcomes. Using college administrative and state workforce data, the authors conducted a nonexperimental study to compare outcomes between students in the PIVA program with a comparison group of non-participating students. The study found no statistically significant relationships between participation in the PIVA program and education or employment outcomes. The quality of causal evidence presented in this report is low because the authors did not ensure the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to PluggedInVA; other factors are likely to have contributed.","PluggedInVA (PIVA)","Capacity building programs Community college education and other classroom training","Adult, Unemployed, Other",,"United States",2016,https://www.skillscommons.org//handle/taaccct/15650,"Community College Review Protocol" "Trade Adjustment Assistance Community College and Career Training II Grant Program (TAACCCT II): Summative Evaluation of the University of District of Columbia, Community College’s Transportation Academy","Yoder, B., & Ryland, A. (2016). Trade Adjustment Assistance Community College and Career Training II Grant Program (TAACCCT II): Summative Evaluation of the University of District of Columbia, Community College’s Transportation Academy. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/15654/University%20of%20the%20District%20of%20Columbia%20-%20Final%20Evaluation%20-%202017.01.05.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Education and skills gains-Low-No impacts Education and skills gains Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the TAACCCT funded Transportation Academy program on education, earnings, and employment outcomes. The study used a nonexperimental design to compare education, earnings, and employment outcomes of Transportation Academy participants to a matched comparison group from three different institutional data sources. The study did not find a significant association between participation in the Transportation Academy program and education, earnings, or employment outcomes. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before intervention. This means we are not confident that the estimated effects are attributable to the Transportation Academy program; other factors are likely to have contributed.","the Transportation Academy Program","Capacity building programs Community college education and other classroom training","Adult, Unemployed, Other",,"United States",2016,https://www.skillscommons.org/bitstream/handle/taaccct/15654/University%20of%20…,"Community College Review Protocol" "Men and women expecting to work longer: Do changing work conditions matter?","Moen, P., Kojola, E., Kelly, E.L., & Karakaya, Y. (2016). Men and women expecting to work longer: Do changing work conditions matter? Work, Aging and Retirement, 2(3), 321-344.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of a work flexibility intervention called STAR (Support. Transform. Achieve. Results) on workers’ expectations of retirement age and bridge jobs. The study is a randomized controlled trial that uses surveys of workers before the intervention and five years post-intervention. The study found that receiving STAR was associated with higher expectations of working at age 65 and retiring at age 67 or older. The quality of causal evidence presented in this report is low because, although it was based on a randomized controlled trial, it had unknown attrition and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to STAR; other factors are likely to have contributed.","STAR (Support. Transform. Achieve. Results.)","Work based and other occupational training Other employer services","Adult, Older worker",,"United States",2016,https://experts.umn.edu/en/publications/men-and-women-expecting-to-work-longer-…,"Employment and Training Review Protocol" "Framing and claiming: How information-framing affects expected social security claiming behavior","Brown, J. R., Kapteyn, A., & Mitchell, O. S. (2016). Framing and claiming: How information-framing affects expected social security claiming behavior. Journal of Risk and Insurance, 83(1), 139-162.","Behavioral Finance: Retirement, Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of different ways of framing retirement information on the age at which individuals intended to claim Social Security benefits. The authors randomly assigned people who had been employed for at least 10 years and had yet to claim Social Security benefits to receive different messages about claiming the benefits. Each message contained the same financial information, but the information was presented—or framed—in different ways. There were 10 different messages—9 treatment conditions and one control condition that presented neutral information. Each study participant received 6 of the 10 messages. The study found that, on average, those presented with the break-even treatment were more likely to indicate a younger intended age to receive benefits compared with those presented with the control condition, whereas those who received 4 of the other treatment conditions expressed an older intended age compared with the control condition. The quality of causal evidence presented in this report is low because it was based on a randomized controlled trial with unknown attrition and the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the framing of information about the age at which Social Security is collected; other factors are likely to have contributed.",,"Federal retirement benefits Behavioral interventions",Employed,,"United States",2016,,"Behavioral Finance: Retirement, Behavioral Insights Review Protocol" "The Green Jobs and Health Care impact evaluation: Findings from the impact study of four training programs for unemployed and disadvantaged workers","Martinson, K., Williams, J., Needels, K., Peck, L., Moulton, S., Paxton, N., Mastri, A., Copson, E., Comfort, A., & Brown-Lyons, M. (2016). The Green Jobs and Health Care impact evaluation: Findings from the impact study of four training programs for unemployed and disadvantaged workers. Retrieved from https://wdr.doleta.gov/research/FullText_Documents/ETAOP-2017-07%20Findings%20from%20the%20Impact%20Study.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment","Summary: The study’s objective was to examine the impact of the Grand Rapids Community College (GRCC) Pathways to Prosperity program on education, earnings, and employment outcomes. The study was a randomized controlled trial. The authors used a baseline information form, the National Directory of New Hires (NDNH), a follow-up survey, and administrative program data to compare the outcomes between the treatment and control groups over an 18-month follow-up period. The study found that the GRCC Pathways to Prosperity program had a significant positive impact on completion rates for vocational training and life skills classes, and receipt of a vocational credential. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the GRCC Pathways to Prosperity program and not to other factors.","the Grand Rapids Community College (GRCC) Pathways to Prosperity Program","Basic skills Capacity building programs Community college education and other classroom training","Adult, Justice-involved, Other barriers, Unemployed, Low income",,"United States",2016,https://wdr.doleta.gov/research/FullText_Documents/ETAOP-2017-07%20Findings%20f…,"Community College Review Protocol" "Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs [Comparison between Full-WIA Services versus Core-and-Intensive Services].","McConnell, S., Fortson, K., Rotz, D., Schochet, P., Burkander, P., Rosenberg, L., Mastri, A., & D’Amico, R. (2016). Providing public workforce services to job seekers: 15-month impact findings on the WIA Adult and Dislocated Worker programs. Washington, DC: Mathematica Policy Research.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Adult and Dislocated Worker programs on training, employment, earnings, and service receipt after 15 months. This summary focuses on the comparison between the group who received the full-WIA services versus the group who received core-and-intensive services. The study was a randomized controlled trial, where customers in each local area were randomly assigned to one of three groups: full-WIA, core-and-intensive, or core. The authors compared the outcomes between the groups from baseline to 15 months after random assignment. The study found that full-WIA customers were significantly more likely than core-and-intensive customers to complete a training program and receive a training credential. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that any estimated effects are attributable to the Adult and Dislocated Worker programs and not to other factors.","Adult and Dislocated Worker programs","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2016,https://www.mathematica-mpr.com/our-publications-and-findings/publications/prov…,"Review Protocol" "Retraining the Gulf Coast through Information Technology Pathways: Final impact evaluation report","Patnaik, A., & Prince, A. (2016). Retraining the Gulf Coast through Information Technology Pathways: Final Impact evaluation report. Austin, TX: The University of Texas at Austin, Lyndon B. Johnson School of Public Affairs.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Retraining the Gulf Coast Workforce through Information Technology Pathways Consortium (Gulf Coast IT Pathways) grant program on education outcomes. Using a nonexperimental design with institutional data from the college systems, the study authors compared students in the program to a matched historical cohort comparison group. The study found that the Gulf Coast IT Pathways program was significantly related to higher credential, certificate, or a degree attainment and higher credit hour accumulation. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Gulf Coast IT Pathways TAACCCT; other factors are likely to have contributed.","the Gulf Coast IT Pathways Program","Basic skills Capacity building programs Community college education and other classroom training Science, Technology, Engineering, and Math (STEM) programs","Adult, Dislocated or displaced worker, Low-skilled, Veteran or military",,"United States",2016,https://raymarshallcenter.org/files/2016/11/Retraining-the-Gulf-Coast-through-I…,"Community College Review Protocol" "The labor market effects of U.S. reemployment programs during the Great Recession","Michaelides, M., & Mueser, P. (2016). The labor market effects of U.S. reemployment programs during the Great Recession. (Working paper 08-2015). Nicosia, Cyprus: University of Cyprus, Department of Economics.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of Florida’s Priority Reemployment Services (PREP) program on Unemployment Insurance (UI) claimants’ reemployment rates, earnings, and receipt of UI benefits. The study was a randomized controlled trial. UI claimants were randomly assigned to either a treatment group that could receive PREP services, another treatment group that could receive Reemployment Eligibility and Assessment (REA) services, or a control group that had access to neither. The authors examined administrative data from Florida’s UI claims and wage records. The study found that the PREP group had significantly higher reemployment rates and lower UI benefit receipt but had similar earnings as the control group. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the PREP program, and not to other factors. The study also examined the impact of Florida’s REA program compared with the control group. CLEAR’s profile of that study is available here.","Profiling, Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2016,http://papers.econ.ucy.ac.cy/RePEc/papers/08-15.pdf,"Reemployment Review Protocol" "A controlled trial of supported employment for people with severe mental illness and justice involvement","Bond, G. R., Kim, S. J., Becker, D. R., Swanson, S. J., Drake, R. E., Krzos, I. M., . . . Frounfelker, R. L. (2015). A controlled trial of supported employment for people with severe mental illness and justice involvement. Psychiatric Services, 66(10), 1027-1034.",Reentry,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Individual Placement and Support (IPS) intervention on employment and recidivism outcomes relative to a job club program called Work Choice. The authors used an experimental design to estimate the impact of IPS and analyzed data from people’s self-reports on forms and interviews and electronic medical and employment records. The authors found positive statistically significant impacts of IPS on employment compared with the job club program. The authors found no statistically significant impacts of IPS on recidivism. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the IPS intervention and not to other factors.","the Individual Placement and Support (IPS) intervention","Health Reentry","Justice-involved, Unemployed",,"United States",2015,https://doi.org/10.1176/appi.ps.201400510,"Employment and Training Review Protocol" "The impact of a randomly assigned time and place management initiative on work and retirement expectations.","Cahill, K. E., James, J. B., & Pitt-Catsouphes, M. (2015). The impact of a randomly assigned time and place management initiative on work and retirement expectations. Work, Aging and Retirement, 1(4), 350-368.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Unfavorable impacts Employment","Summary: The study examined the impact of a time and place management (TPM) initiative at a medical provider on retirement expectations among workers ages 50 and older. The study was a randomized control trial and used a series of surveys and administrative data to estimate the impact. The study found that, when surveyed 12 months after randomization, employees in the treatment group were statistically significantly more likely to expect to directly exit their job as opposed to staying in their current job within the next five years. The quality of causal evidence presented in this report is moderate because the study was a randomized controlled trial with high attrition, but the authors did demonstrate that the treatment and control groups were similar before the intervention. This means we are somewhat confident that the estimated effects are attributable to TPM, but other factors might also have contributed.","Time and Placement Management (TPM)","Older workers' programs","Older worker","Health care and social assistance",,2015,https://hrs.isr.umich.edu/publications/biblio/8231,"Employment and Training Review Protocol" "Becoming adults: One year impact findings from the youth villages’ Transitional Living evaluation","Valentine, E.J., Skemer, M., & Courtney, M.E. (2015). Becoming adults: One year impact findings from the youth villages’ Transitional Living evaluation. New York: MDRC.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Low-No impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Transitional Living program on education, earnings, and recidivism outcomes. The authors used a randomized controlled trial and estimated impacts by comparing the means of the treatment and control groups. Outcome data on education, earnings, and criminal involvement came from a 12-month survey conducted by the authors. The study found a statistically significant impact on earnings only. Youth in the treatment group earned an average $611 more than youth in the control group. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Transitional Living, and not to other factors.","Transitional Living","Basic skills Other training and education Youth programs","Youth, Justice-involved, Disconnected youth",,"United States",2015,http://www.mdrc.org/sites/default/files/Becoming_Adults_FR.pdf,"Opportunities for Youth Review Protocol" "Learning about job search: A field experiment with job seekers in Germany","Altmann, S., Falk, A., Jäger, S., & Zimmermann, F. (2015). Learning about job search: A field experiment with job seekers in Germany. Unpublished manuscript.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment","Summary: The study’s objective was to examine whether providing job search information to unemployed job seekers affected their labor market outcomes. German citizens identified as unemployed were randomly assigned to the treatment group, which was mailed an informational brochure, or to the control group, which was not mailed a brochure. The Integrated Employment Biographies, an administrative database available through the German Federal Employment Agency, was the primary data source for the study. The study did not find any statistically significant effects of the brochure on the employment or earnings outcomes of the treatment group relative to the control group. However, the study did find positive impacts of the brochure on earnings and employment outcomes for the subgroup of individuals who had been identified as being at risk of long-term unemployment. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the informational brochure, and not to other factors.",,"Job search assistance and supportive services Behavioral interventions","Adult, Unemployed",,International,2015,https://www.cens.uni-bonn.de/team/board/armin-falk/afjz-job-search-dp-150509.pdf,"Behavioral Insights Review Protocol" "Employment outcomes among African American and white women with disabilities: Examining the inequalities.","Balcazar, F. E., & Suarez-Balcazar, Y. (2015). Employment outcomes among African American and white women with disabilities: Examining the inequalities. Women, Gender, and Families of Color, 3(2), 144-164.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment","Summary: The study examined the impact of adult vocational rehabilitation (VR) services on employment outcomes for African American and white women. The authors used a statistical model and data from a Midwestern state VR system to compare the odds of employment after completing VR services for older versus younger women. The study found no significant difference in employment outcomes between older and younger women who completed VR services. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before receiving VR services. This means we are not confident that the estimated effects are attributable to participation in VR services; other factors are likely to have contributed.","Vocational Rehabilitation (VR) Services","Vocational rehabilitation","Adult, Older worker, Female, Black or African American, White, Disability",,"United States",2015,,"Employment and Training Review Protocol" "The effect of health reform on retirement.","Levy, H., Buchmueller, T. C., & Nikpay, S. (2015). The effect of health reform on retirement (Research Paper 329). Ann Arbor, MI: Michigan Retirement Research Center.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the expansion of Medicaid to low-income adults in 2014 on retirement rates of workers ages 55 to 64. The authors used a nonexperimental analysis to compare those who lived in an expansion state to those who lived in other states. The study used data from the monthly Current Population Survey from January 2005 through July 2015. The study found no statistically significant relationships between Medicaid expansion and retirement. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the expansion of Medicaid; other factors are likely to have contributed.","Medicaid expansion","Health insurance Older workers' programs","Adult, Older worker, Low income",,"United States",2015,https://deepblue.lib.umich.edu/handle/2027.42/116062,"Employment and Training Review Protocol" "Job rationing in recessions: evidence from work-search requirements.","Toohey, D. (2015). Job rationing in recessions: evidence from work-search requirements. Newark, Delaware: University of Delaware.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment Public benefits receipt-Low-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of required contacts with employers on Unemployment Insurance (UI) claimants’ employment and UI benefit receipt. The study used a nonexperimental design in which the author compared the labor market outcomes of UI claimants in different states who were subject to different job search requirements between 2001 and 2013. The study found that an increase in the number of required contacts with employers was associated with decreases in the unemployment rate. Accounting for expected employment growth along with changes in the number of required contacts also decreased the unemployment rate. The study found no statistically significant relationships between the number of required contacts and UI claim duration. The quality of causal evidence presented in this report is low because the author did not account for differences in the age of UI claimants in the analysis. This means we are not confident that the estimated effects are attributable to required contacts with employers; other factors are likely to have contributed.","More stringent work search requirements","RESEA Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2015,,"Reemployment Review Protocol" "Can compulsory dialogues nudge sick-listed workers back to work?","Markussen, S., Røed, K., & Schreiner, R. C. (2015). Can compulsory dialogues nudge sick-listed workers back to work? (IZA Discussion Paper No. 9090). Bonn, Germany: Institute for the Study of Labor (IZA).","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of a compulsory dialogue meeting (DM) on the return to partial or full-time employment for long-term sick-listed workers in Norway. The authors compared differences in outcomes between long-term sick-listed workers whose absence started in July 2004 through June 2006 and those whose absence started in January 2009 through December 2010, during which time the DM program became compulsory. The authors used salary and social insurance data from the Norwegian Social Security Administration (SSA) to measure program impacts. The study found that increasing the probability of a sick-listed worker attending a DM resulted in a significant increase in base salary, for those who returned to full-time employment, of $1,436. It also resulted in a significantly lower probability of receiving temporary disability benefits two years later for those who returned to full-time employment (1.4 percent) and those who returned to either part- or full-time employment (2.3 percent). The quality of causal evidence presented in this report is low because the authors did not establish the comparability of the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to compulsory DMs; other factors are likely to have contributed.","Dialogue Meeting (DM)","Behavioral Interventions","Unemployed, Adult",,International,2015,http://ftp.iza.org/dp9090.pdf,"Behavioral Insights Review Protocol" "Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment","Bhargava, S., & Manoli, D. (2015). Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment. American Economic Review, 105(11), 3489-3529.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of complexity, program information, and stigma on the take-up rate of the Earned Income Tax Credit (EITC) among eligible taxpayers who had not previously claimed the credit. The authors conducted a randomized controlled trial, sending different combinations of materials to likely eligible taxpayers who had not previously claimed the EITC. They used tax data from the Internal Revenue Service (IRS) for tax year 2009. The study found that use of complex materials and a longer worksheet decreased the EITC take-up rate by 6 and 4 percentage points, respectively. Materials that displayed the potential tax credit amount increased the EITC take-up rate by 8 percentage points. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the experimental mailings, and not to other factors.",,"Behavioral Interventions","Adult, Employed",,"United States",2015,,"Behavioral Insights Review Protocol" "The effect of a severe health shock on work behavior: Evidence from different health care regimes.","Datta Gupta, N., Kleinjans, K. J., & Larsen, M. (2015). The effect of a severe health shock on work behavior: Evidence from different health care regimes. Social Science and Medicine, 136-137, 44-51.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the national health policies of the United States and Denmark on employment of older workers after a health shock, defined as a new cancer or cardiovascular disease diagnosis. The authors used a nonexperimental approach to compare the impact of a health shock on the likelihood of nonemployment in the United States’ private multipayer system with Denmark’s nationalized universal health care model. The authors used the data from the RAND Health and Retirement Study survey from 1994 to 2002 and Danish population registers for 1993 to 2001. The study found no differences between the United States and Denmark in the relationship between a health shock and employment for older workers. The quality of causal evidence presented in this report is low because the authors did not account for unobserved differences between the countries. This means we are not confident that differences in the association between health shocks and employment in the United States and Denmark are attributable to the national health care policies; other factors are likely to have contributed.","National Healthcare Policy","Health Older workers' programs","Adult, Older worker",,"International, United States",2015,https://www.sciencedirect.com/science/article/pii/S0277953615002816,"Employment and Training Review Protocol" "How much does access to health insurance influence the timing of retirement?","Coe, N. B., & Goda, G. S. (2015). How much does access to health insurance influence the timing of retirement? (SIEPR Discussion Paper No. 14-007). Stanford, CA: Stanford Institute for Economic Policy Research. City, ST: Publisher.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment Public benefits receipt-Low-No impacts Public benefit receipt","Summary: The study examined the impact of state-level reforms of health insurance on early retirement and Social Security retirement benefit claims. The authors used 1996–2010 data from the Health and Retirement Study (HRS) and statistical models to compare the outcomes of individuals in states with state-level reforms of health insurance with those in non-reform states. The study found that employed individuals in states with non-group health insurance reform were significantly more likely to retire at age 63. The quality of causal evidence presented in this report is low because the authors did not demonstrate that individuals in states with state-level reforms of health insurance were similar to those in non-reform states. This means we are not confident that the estimated effects are attributable to the non-group health insurance reform; other factors are likely to have contributed.","Retiree Health Insurance","Health insurance Older workers' programs","Adult, Older worker",,"United States",2015,,"Employment and Training Review Protocol" "The effects of eliminating the work search requirement on job match quality and other long-term employment outcomes.","Lachowska, M., Meral, M., & Woodbury, S.A. (2015). The effects of eliminating the work search requirement on job match quality and other long-term employment outcomes. Washington, DC: U.S. Department of Labor.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-Unfavorable impacts Employment Public benefits receipt-Mod/high-Unfavorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of less-stringent work search requirements for unemployment insurance (UI) claimants on earnings, employment, and UI benefit receipt over a nine-year follow-up period. This study extended the work of Johnson and Klepinger (1991), the CLEAR profile of which is available here. The CLEAR profile of a related study, Lachowska et al. (2016), is available here. The study examined data from a randomized controlled trial conducted in the 1980s in Washington State. UI claimants were randomly assigned to one of four groups, each with different work search requirements and/or verification of that work search. The authors used administrative data to compare the outcomes of the groups up to nine years after the initial UI claims. The study found that the group with less-stringent work-search requirements was significantly less likely to be employed in the first quarter following their claims, compared with the groups with more-stringent requirements. In addition, they received more UI benefit payments for more weeks and exhausted UI benefits at a higher rate during the year following their initial claims. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the less-stringent work search requirements, and not to other factors.","Less stringent work search requirements","RESEA Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2015,https://www.dol.gov/asp/evaluation/completed-studies/2013-2014-scholar-programs…,"Reemployment Review Protocol" "Evaluation of MoHealthWINS outcomes and impact","Cosgrove, J. J., Cosgrove, M. S., & Bragg, D. D. (2015). Evaluation of MoHealthWINS outcomes and impact. Retrieved from https://www.skillscommons.org/bitstream/handle/taaccct/5125/SEPTEMBER%202015%20FINAL%20VERSION%20OF%20MHW%20OUTCOME%20IMPACT%20EVALUATION%20REPORT.pdf?sequence=1&isAllowed=y","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to assess the relationship between the MoHealthWINS program and education and employment outcomes. The study used a nonexperimental design, where the program participants were compared to a matched comparison group. Using college administrative data, state employment data, and employer data, the authors compared the program completion and employment rates between the treatment and comparison group. The study found a significant relationship between program participation and the likelihood to complete the program and be employed post-program completion. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient control variables. This means we are not confident that the estimated effects are attributable to MoHealthWINS program; other factors are likely to have contributed.",MoHealthWINS,"Capacity building programs Community college education and other classroom training","Adult, Unemployed, Low-skilled, Veteran or military",,"United States",2015,https://www.skillscommons.org/bitstream/handle/taaccct/5125/SEPTEMBER%202015%20…,"Community College Review Protocol" "Net impact study on annual earnings for the training benefits program 2002-2012","Aviles, G., Bordelon, C., Greenwell, A., Pham, L., Sinkler, A., & Stromsdorfer, E. (2015). Net impact study on annual earnings for the training benefits program 2002-2012. Olympia, WA: Washington State Employment Security Department",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment Public benefits receipt-Mod/high-Mixed impacts Public benefit receipt","Summary: The study’s objective was to evaluate the impact and cost-benefit of the Training Benefits program on participants’ employment, earnings, and unemployment benefits. The study used a nonexperimental design to compare the outcomes of individuals who participated in the program with those who did not, based on data from the Washington Employment Security Department. Using several demographic characteristics, the authors created a matched comparison group to test the impact of the program on time employed, annual earnings, and receipt of unemployment benefits. The study found that program participation was associated with a decrease in employment and earnings in early follow up years, but an increase in employment and earnings in later years. However, the study did not include tests of statistical significance. The quality of causal evidence provided in this study is moderate because it is based on a well-implemented non-experimental design. This means we are somewhat confident that the estimated effects are attributable to the Training Benefits program, but other factors might have also contributed.","Training Benefits (TB) Program","Unemployment Insurance Work based and other occupational training","Adult, Unemployed",,"United States",2015,,"Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington state. (Upjohn Institute Technical Report No. 13-029). [WIA Dislocated Workers]","Hollenbeck, K., & Huang, W-J. (2014). Net impact and benefit-cost estimates of the workforce development system in Washington state. (Upjohn Institute Technical Report No. 13-029). Retrieved from W.E. Upjohn Institute for Employment Research website: http://dx.doi.org/10.17848/tr13-029%20[WIA Dislocated Workers]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Dislocated Worker Program on the employment rate, earnings, and benefit receipt of dislocated workers in Washington State. The authors assigned dislocated workers to the treatment group if they received employment-related services through the WIA Dislocated Worker Program and exited from July 2005 to June 2006 or from July 2007 to June 2008. The comparison group comprised workers who registered for services at the Labor Exchange during this period. The study found that in both the short and long term the employment rate and average quarterly earnings increased for those in the treatment group compared to those in the comparison group. Benefit receipt decreased for those in the treatment group compared with those in the comparison group. The quality of causal evidence presented in this report is low because the treatment and comparison group were compared at different follow-up points and therefore were not equivalent. This means we are not confident that the estimated effects are attributable to the WIA Dislocated Worker Program; other factors are likely to have contributed.","the Workforce Investment Act (WIA) Dislocated Worker Program","Job search assistance and supportive services Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker",,"United States",2014,http://research.upjohn.org/cgi/viewcontent.cgi?article=1032&context=up_technica…,"Employment and Training Review Protocol" "High-involvement work practices and the opportunities for promotion in the organization","Bonet, R. (2014). High-involvement work practices and the opportunities for promotion in the organization. Industrial Relations, 53(2), 295-324. doi:10.1111/irel.12057","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of employees’ participation in high-involvement work practices (HIWP) on their expected promotions. The author used regression methods in a nonexperimental analysis to estimate impacts of participating in HIWP, drawing on self-reported data from a survey of for-profit company employees. The study found that employee participation in more HIWP was associated with a greater likelihood of expected promotions. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to participation in HIWP, but other factors might also have contributed.","High-Involvement Work Practices (HIWP)","Work based and other occupational training","Adult, Employed",,"United States",2014,https://doi.org/10.1111/irel.12057,"Employment and Training Review Protocol" "Training program impacts and the onset of the Great Recession. [Adult sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [Adult sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages","Summary: The study’s objective was to examine whether participation in the Workforce Investment Act (WIA) Adult Program services increased participants' earnings and if so, whether those gains were greater during a recessionary period than in periods before and after the recession. The authors analyzed administrative data to compare the quarterly earnings of participants receiving services through the WIA Adult Program with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year. The study found that males and females receiving services from the WIA Adult Program from June to December 2007 (before the recession) earned more, on average, than the comparison group in some, but not all, of the 16 follow-up quarters examined. Males and females receiving services from July 2008 to June 2009 and from July 2009 to June 2010 (after the onset of the recession) tended to earn less, on average, than the comparison group in most of the 16 follow-up quarters. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Adult Program services, but other factors might also have contributed. The report also examined the impacts of WIA Dislocated Worker services; that profile is available here.","the Workforce Investment Act (WIA) Adult Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.missourieconomy.org/pdfs/WDQI_MissouriReport_TrainingProgramImpacts…,"Review Protocol" "Influential leadership in a diverse retail environment: implications for reducing voluntary employee turnover","Jackson, S. (2014). Influential leadership in a diverse retail environment: implications for reducing voluntary employee turnover. (Doctoral dissertation). Retrieved from ProQuest. Accession No. 1640901590.","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of a retail store management intervention on voluntary employee turnover. The author used a nonexperimental design to compare the rate of voluntary turnover of employees in stores whose management received the intervention with the rate of stores in the same retail chain whose management did not. The author estimated impacts using store administrative data on employee turnovers as well as an employee opinion survey. The study found that treatment group stores experienced a decrease in voluntary employee turnover and that the comparison group stores did not. The author did not test this outcome statistically. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the retail store management intervention; other factors are likely to have contributed.","the retail store management intervention","Other training and education",Employed,,"United States",2014,https://scholarworks.waldenu.edu/cgi/viewcontent.cgi?referer=https%3A//scholar…,"Employment and Training Review Protocol" "Training program impacts and the onset of the Great Recession. [TAA sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [TAA sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts Earnings and wages","Summary: The study’s objective was to examine whether participation in the Trade Adjustment Assistance (TAA) Program services increased participants’ earnings and, if so, whether those gains were greater during a recessionary period than in periods before and after the recession. The authors analyzed administrative data to compare the quarterly earnings of TAA participants with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year. The study found that males and females who received services through the TAA Program earned less, on average, than the comparison group during almost all of the 16 follow-up quarters examined. This held regardless of whether the participants began receiving services before, during, or after the recession. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to services received through the TAA Program, but other factors might also have contributed.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other training and education","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.empiwifo.uni-freiburg.de/sonstige-dateien/heinrich-mueser_trainingi…,"Review Protocol" "Work-related CBT versus vocational services as usual for unemployed persons with social anxiety disorder: A randomized controlled pilot trial.","Himle, J., Bybee, D., Steinberger, E., Laviolette, W., Weaver, A., Vlnka, S., Golenberg, Z., Levine, D., Heimberg, R., & O’Donnell, L. (2014). Work-related CBT versus vocational services as usual for unemployed persons with social anxiety disorder: A randomized controlled pilot trial. Behaviour Research and Therapy, 63, 169-176.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the effect of group-based, work-related cognitive-behavioral therapy (CBT) on work-related outcomes of unemployed, homeless people diagnosed with social anxiety disorder. The study was a randomized controlled trial. The authors randomly assigned eligible participants to either a treatment group, which received work-related CBT and vocational services, or to a control group, which received vocational services alone. Trained independent evaluators collected the self-reported data used in the study. The study found that participants assigned to the treatment group had higher job search self-efficacy and activity than participants assigned to the control group. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects on employment are attributable to work-based CBT; other factors are likely to have contributed.",,"Job search assistance and supportive services","Unemployed, Low income, Homeless",,"United States",2014,,"Employment and Training Review Protocol" "Training program impacts and the onset of the Great Recession. [Dislocated worker sample]","Heinrich, C.J., and Mueser, P. (2014). Training program impacts and the onset of the Great Recession. Columbia, MO: Department of Economics, University of Missouri-Columbia, unpublished. [Dislocated worker sample]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages","Summary: The study’s objective was to examine whether participation in the Workforce Investment Act (WIA) Dislocated Worker Program services increased participants' earnings and if so, whether those gains were greater during a recessionary period than in periods before and after the recession. The authors analyzed administrative data to compare the quarterly earnings of participants receiving services through the WIA Dislocated Worker Program with those of a matched comparison group that received Wagner-Peyser Employment Services (ES). The study presented impacts separately by gender and program year. The study found that males receiving services through the WIA Dislocated Worker Program from July 2007 to December 2007 (before the recession), from January 2008 to June 2008 (at the beginning of the recession), and from July 2008 to June 2009 (during the worst of the recession) earned less than males in the comparison group in most of the 16 follow-up quarters examined. In no quarter did females participating in the WIA Dislocated Worker Program before, during, or after the recession earn more than females in the comparison group. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Dislocated Worker Program services, but other factors might also have contributed. The report also examined the impacts of WIA Adult Program services; that profile is available here.","the Workforce Investment Act (WIA) Dislocated Worker Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs Job search assistance and supportive services","Dislocated or displaced worker, Female, Male",,"United States",2014,https://www.missourieconomy.org/pdfs/WDQI_MissouriReport_TrainingProgramImpacts…,"Review Protocol" "Did age discrimination protections help older workers weather the Great Recession?","Neumark, D., & Button, P. (2014). Did age discrimination protections help older workers weather the Great Recession? Journal of Policy Analysis and Management, 33(4), 566-601. doi:10.1002/pam.21762","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Unfavorable impacts Employment","Summary: The study examined whether the strong age discrimination laws moderated the impact of the Great Recession on employment outcomes of older workers. The study used statistical models and the data from the 2003–2011 Current Population Survey (CPS) and 2004–2011 Quarterly Workforce Indicators to estimate impacts. The study found that states with stronger age discrimination laws had experienced increases in unemployment rates, longer unemployment durations, and decreases in hiring rates for some types of older workers relative to younger workers in the months during and following the Great Recession. The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the differential effect of the Great Recession on older workers in states with strong age discrimination laws, but other factors might also have contributed.","Age Discrimination Laws","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Female, Male",,"United States",2014,https://www.nber.org/papers/w19216,"Employment and Training Review Protocol" "Reference-dependent job search: Evidence from Hungary","DellaVigna, S., Lindner, A., Reizer, B., & Schmieder, J. (2014). Reference-dependent job search: Evidence from Hungary. Unpublished working paper.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to empirically test a behavioral model of job search using a reform in Hungary’s Unemployment Insurance (UI) program. The authors compared rates of exit from unemployment in groups of claimants who entered UI before and after the reform’s implementation in November 2005. The authors used social security and UI data from Hungary’s Institute of Economics and National Employment Service. The study found support for the behavioral job search model and theorized that approximately budget-neutral transitions to two-step UI systems could speed claimants’ exit out of unemployment. The quality of causal evidence presented in this report is low because it cannot account for factors other than the UI program that also changed over the period of study and could also have influenced the outcomes of interest. This means we are not confident that the estimated effects are attributable to Hungary’s 2005 UI reform; other factors are likely to have contributed.","Hungary’s Unemployment Insurance Reform","Unemployment Insurance Behavioral Interventions Job search assistance and supportive services","Adult, Unemployed, Dislocated or displaced worker",,International,2014,http://eml.berkeley.edu/~sdellavi/wp/HungaryUIRD_wp_2014_07_20.pdf,"Behavioral Insights Review Protocol" "Retiree health insurance for public school employees: Does it affect retirement?","Fitzpatrick, M. D. (2014). Retiree health insurance for public school employees: Does it affect retirement? Journal of Health Economics, 38, 88-98.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of an Illinois Public Schools retiree health insurance program on the retirement rates of eligible staff ages 55 to 75. The study used a nonexperimental approach to compare the retirement rates of eligible staff with those of staff who were not eligible and staff in years prior to the introduction of the health insurance program who would have been eligible. The author used data from the Illinois Public Schools' Teacher Service Record database from 1970 to 1992. The study found that eligibility for the program was associated with an overall decrease in retirement age, with eligible staff more likely to retire at ages 55 or 56, but less likely to retire after age 65 compared with those who were not eligible for the program. The quality of the causal evidence presented in this report is low because the study is a nonexperimental analysis that does not account for possible differences in the groups. This means we are not confident that the estimated effects are attributable to the Illinois Public Schools Teachers Retirement Health Insurance Plan; other factors are likely to have contributed.","Illinois Public Schools Teachers Retirement Health Insurance Plan","Employer provided retirement benefits Health insurance Older workers' programs","Adult, Older worker",,"United States",2014,https://www.nber.org/papers/w19524,"Employment and Training Review Protocol" "The economics of federal tax and employment laws: Empirical evidence from three studies.","Figinski, T. F. (2013). The economics of federal tax and employment laws: Empirical evidence from three studies. (Doctoral dissertation). Retrieved from ProQuest. Accession No. 1415455735.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Unfavorable impacts Public benefit receipt","Summary: The study examined the impact of removing the Social Security earnings test on Social Security claims, earnings, and labor force participation for female beneficiaries. The study used a statistical model and the data from the Social Security Administration’s Benefit and Earnings Public Use File to estimate impacts. The study found that removing the earnings test was associated with an increase in Social Security claims among women who were primary or spousal beneficiaries and with an increase in average earnings and in labor force participation among women who were primary beneficiaries. The qaulity of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the change in Social Security earings test. This means we are not confident that the estimated effects are attributable to the removal of the Social Security earnings test; other factors are likely to have contributed.","Social Security 2000 Earnings Test Removal","Federal retirement benefits Older workers' programs","Adult, Older worker, Female",,"United States",2014,,"Employment and Training Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington state. (Upjohn Institute Technical Report No. 13-029). [Community and Technical College Worker Retraining Program]","Hollenbeck, K., & Huang, W-J. (2014). Net impact and benefit-cost estimates of the workforce development system in Washington state. (Upjohn Institute Technical Report No. 13-029). Retrieved from W.E. Upjohn Institute for Employment Research website: http://dx.doi.org/10.17848/tr13-029%20 [Community and Technical College Worker Retraining Program]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Community and Technical College (CTC) Worker Retraining (WR) program on employment, earnings, and benefit receipt of long-term unemployed and dislocated workers who are eligible for or have exhausted their unemployment benefits in Washington State. The authors assigned workers to the treatment group if they received training through the CTC WR program and exited from July 2005 to June 2006 or from July 2007 to June 2008. The comparison group comprised workers who registered at the Labor Exchange. The study found that both short- and long-term employment and earnings increased for those who participated in the WR program compared to those in the Labor Exchange. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the CTC WR program; other factors are likely to have contributed.","Community and Technical College Worker Retraining (CTC WR) Program","Other training and education","Dislocated or displaced worker",,"United States",2014,http://research.upjohn.org/cgi/viewcontent.cgi?article=1032&context=up_technica…,"Employment and Training Review Protocol" "Who pays the high health costs of older workers? Evidence from prostate cancer screening mandates.","Bailey, J. (2014). Who pays the high health costs of older workers? Evidence from prostate cancer screening mandates. Applied Economics, 46(32), 3931-3941. doi:10.1080/00036846.2014.948673","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts Earnings and wages Employment-Mod/high-Unfavorable impacts Employment","Summary: The study examined the impact of state mandates that health insurance plans cover prostate cancer screenings on the earnings and employment of men older than age 50 The study used nonexperimental regression analyses to compare older men in states that had enacted a mandate with women, younger men, and older men living in states without a mandate. The author used data from the Current Population Survey(CPS) between 1990 and 2009. The study found that older men in states with mandates had lower wages and were less likely than the comparison group to be employed. The quality of causal evidence presented in this study is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to the prostrate screening mandate, but other factors might also have contributed.","Prostate cancer screening mandates","Health insurance Older workers' programs","Adult, Older worker, Male",,"United States",2014,https://sites.temple.edu/jamesbailey/files/2013/08/Who-Pays-the-High-Health-Cos…,"Employment and Training Review Protocol" "Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms","Andersson, F., Holzer, H. J., Lane, J. I., Rosenblum, D., & Smith, J. (2013). Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms (Discussion paper no. 7621). Bonn, Germany: IZA. [Adult Program ONLY]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) adult program’s training services on the employment and earnings of low-income adults. The authors investigated similar research questions in another study, whose profile can be found [here]. The other study examined the effects of WIA’s dislocated worker program’s training services on the employment and earnings of dislocated workers. The authors used statistical analysis to compare the earnings and employment outcomes of WIA-registered low-income adults who received training services to the outcomes of those who were registered in WIA, but did not receive training services. The study found that employment was significantly lower for the treatment group than it was for the comparison group in the first quarters after WIA registration (Quarters 1 and 2 in State A and Quarters 1–4 in State B), but later became and generally stayed significantly higher in Quarters 6 through 12 in both states. Total earnings in the third year after WIA registration were significantly higher for the treatment group than the comparison group. People in the treatment group earned $1,257 and $1,703 more than those in the comparison group in State A and State B, respectively. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA adult program training services, but other factors might also have contributed to the effects.","Workforce Investment Act (WIA) Program Training Services","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Unemployed, Low income",,"United States",2013,https://www.nber.org/papers/w19446.pdf,"Employment and Training Review Protocol" "Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms. [Dislocated Workers ONLY]","Andersson, F., Holzer, H. J., Lane, J. I., Rosenblum, D., & Smith, J. (2013). Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms (Discussion paper no. 7621). Bonn, Germany: IZA. [Dislocated Workers ONLY]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) dislocated worker program’s training services on the employment and earnings of dislocated workers. The authors used a regression model with inverse propensity weights, to compare the earnings and employment outcomes of WIA-registered dislocated workers who received training services to the outcomes of those who were registered in WIA but did not receive training services. The study found that the treatment group earned significantly less than the comparison group in the three years after WIA registration. Individuals in the treatment group earned an average of $5,567 and $5,227 less than those in the comparison group in State A and State B, respectively. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the training services received, but other factors might also have contributed","Workforce Investment Act (WIA) Program Training Services","Adult and Dislocated Worker programs Work based and other occupational training Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker",,"United States",2013,http://ftp.iza.org/dp7621.pdf,"Review Protocol" "The link between pensions and retirement timing: Lessons from California teachers.","Brown, K. M. (2013). The link between pensions and retirement timing: Lessons from California teachers. Journal of Public Economics, 98(C), 1-14.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of pension reform under the California State Teachers’ Retirement System (CalSTRS) on retirement timing. The author used a statistical model and data from retirement-eligible employees in Los Angeles Unified School District from 1997 to 2000 to estimate the impact. The study found public school employees were more likely to work an extra year in response to changes in pension plans that provided additional bonuses tied to age and tenure. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the changes in CalSTRS. This means we are not confident that the estimated effects are attributable to the changes in CalSTRS; other factors are likely to have contributed.","CalSTRS Defined Benefit Program","Older workers' programs Other wages and benefits","Adult, Older worker",,"United States",2013,https://eml.berkeley.edu//~saez/course/brown_jpube13.pdf,"Employment and Training Review Protocol" "Training and retirement patterns.","Montizaan, R., Cörvers, F., & de Grip, A. (2013). Training and retirement patterns. Applied Economics, 45(15), 1991-1999. doi:10.1080/00036846.2011.646066","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the impact of firm-specific training and general training on older men’s retirement decisions The study used a nonexperimental design and data from the U.S. National Longitudinal Survey of Older Men for 1966 to 1983 to estimate impacts. The study found that workers with firm-specific training were more likely to retire within the time period studied and to have jobs with a mandatory retirement age than workers without firm-specific training. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before receiving training and did not account for self-selection into training. This means we are not confident that the estimated effects are attributable to firm-specific or general training; other factors are likely to have contributed.","Firm-specific & General Training","Work based and other occupational training","Adult, Older worker, Male",,"United States",2013,https://www.researchgate.net/publication/236021967_Training_and_retirement_patt…,"Employment and Training Review Protocol" "The impact of uncapping of mandatory retirement on postsecondary institutions.","Weinberg, S. L., & Scott, M. A. (2013). The impact of uncapping of mandatory retirement on postsecondary institutions. Educational Researcher, 42(6), 338-348.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of uncapping mandatory retirement on retirement ages at postsecondary institutions The study used a statistical model to examine the age of retirement before and after the policy change in 1993. It used data on faculty’s employment and retirement from a research university with multiple schools. The study found that uncapping was associated with a statistically significant increase in the retirement age among those who retired from the university. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar nor did they control for differences between the groups. This means we are not confident that the estimated effects are attributable to the federal Age Discrimination in Employment Act (ADEA); other factors are likely to have contributed.","Age Discrimination Employment Act (ADEA)","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2013,https://www.researchgate.net/publication/258134776_The_Impact_of_Uncapping_of_M…,"Employment and Training Review Protocol" "Preventing youth violence and dropout: A randomized field experiment (No. w19014)","Heller, S., Pollack, H. A., Ander, R., & Ludwig, J. (2013). Preventing youth violence and dropout: A randomized field experiment (No. w19014). Cambridge, MA: National Bureau of Economic Research.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Education and skills gains-Mod/high-Favorable impacts Education and skills gains","Summary: The study’s objective was to examine the impact of the Becoming a Man program on arrest and educational outcomes. The study was a randomized controlled trial. Eligible male youth from 18 public schools in disadvantaged neighborhoods were screened for eligibility. The 2,740 who met requirements were randomly assigned to one of three treatment groups or a control group. The study found that being offered the opportunity to participate in the program reduced violent crime arrests during the program year and improved schooling outcomes in both the program and follow-up years. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Becoming a Man program, and not to other factors.","Becoming a Man","Other training and education Safety Youth programs Behavioral Interventions","Youth, Disconnected youth, Male, Low income",,"United States, Urban",2013,http://faculty.chicagobooth.edu/workshops/AppliedEcon/past/pdf/jensludwigyouthv…,"Opportunities for Youth Review Protocol" "Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females","Dougherty, T. W., Dreher, G. F., Arunachalam, V., & Wilbanks, J. E. (2013). Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females. Journal of Vocational Behavior, 83(3), 514-527. [Study 2, Contrast 2: non-senior mentor versus no mentor]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages","Summary: The study’s objective was to examine the role of informal mentoring on annual compensation, as well as the moderating effects of mentor status. The authors investigated similar research questions in other studies, the profiles of which are available [here]. The authors used statistical methods to analyze self-reported data on earnings, mentoring relationships, and employee characteristics collected through questionnaires administered to the software engineering department of a major aerospace manufacturing firm. The study found no statistically significant differences between the earnings of employees with non-senior mentors and those with no mentors. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that any estimated effects would be attributable to informal mentoring; however, the study did not find any statistically significant effects.",Mentoring,"Other wages and benefits Mentoring","Adult, Employed",,"United States",2013,https://doi.org/10.1016/j.jvb.2013.08.001,"Employment and Training Review Protocol" "Youth Initiated Mentoring: Investigating a new approach to working with vulnerable adolescents","Schwartz, S., Rhodes, J., Spencer, R., & Grossman, J. (2013). Youth Initiated Mentoring: Investigating a new approach to working with vulnerable adolescents. American Journal of Community Psychology, 52, 155–169.","Opportunities for Youth","Study Type: Descriptive Analysis",,,"Summary: The study examined Youth Initiated Mentoring (YIM) within the context of the National Guard Youth ChalleNGe Program (NGYCP). Unlike a traditional mentoring program in which mentors are assigned to youth, under YIM, youth nominate their own mentors. The authors conducted a quantitative analysis using data from the 9-, 21- and 38-month follow-up surveys administered as part of a national evaluation of NGYCP, along with qualitative analysis of interviews with a subsample of NGYCP participants. The study found that YIM mentors included family friends, extended family members and godparents, school and extracurricular staff, and religious leaders. By the 38-month follow-up survey, 56 percent of youth reported still being in contact with their mentors. In addition, NGYCP youth who were in contact with their mentors at the 38-month follow-up showed significant benefits on a range of outcomes compared with the control group.","Youth Initiated Mentoring in the National Guard Youth ChalleNGe","Mentoring Youth programs","Youth, Disconnected youth",,"United States",2013,,"Opportunities for Youth Review Protocol" "Does retiree health insurance encourage early retirement?","Nyce, S., Schieber, S. Shoven, J. B., Slavov, S., & Wise, D. A. (2013). Does retiree health insurance encourage early retirement? Journal of Public Economics, 104, 40-51.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the impact of employer-provided retiree health insurance on early retirement. The authors used a nonexperimental design to compare early retirement outcomes of employees who are eligible for retiree health insurance with that of those who have no coverage. The study found that people who were employed at age 57 and eligible for employer-subsidized retiree health coverage were significantly more likely to retire at ages 62 to 64 (before becoming eligible for Medicare at age 65) compared with those with no coverage. The quality of causal evidence presented in this report is low because the authors did not account for other factors that could have affected the difference between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to retiree health insurance; other factors are likely to have contributed.","Retiree Health Insurance","Employer provided retirement benefits Older workers' programs","Adult, Older worker",,"United States",2013,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3770310/,"Employment and Training Review Protocol" "Stimulating opportunity: An evaluation of ARRA-funded subsidized employment programs","Roder, A. & Elliott, M. (2013). Stimulating opportunity: An evaluation of ARRA-funded subsidized employment programs. New York: Economic Mobility Corporation. Retrieved from http://economicmobilitycorp.org/uploads/stimulating-opportunity-full-report.pdf.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of subsidized employment programs in five states on the employment and earnings outcomes of unemployed people. The focus of this profile is Florida’s Back to Work program, which placed participants in partially subsidized jobs in which employers were expected to cover 5 to 20 percent of the employee’s wages for up to one year. For the evaluation of the Florida Back to Work program, the authors used administrative wage data to compare the employment and earnings outcomes of program participants with the employment and earnings outcomes of a similar group of people who were not placed in subsidized employment. The study found that participants in Florida Back to Work were more likely to be employed and earned $1,921 more in the year following the program than those in the comparison group. The quality of causal evidence presented in this report is low because the authors did not include sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the subsidized employment program. Other factors are likely to have contributed.","Florida's Back to Work Program","Other employment and reemployment","Unemployed, Low income",,"United States",2013,http://economicmobilitycorp.org/uploads/stimulating-opportunity-full-report.pdf,"Employment and Training Review Protocol" "Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females","Dougherty, T. W., Dreher, G. F., Arunachalam, V., & Wilbanks, J. E. (2013). Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females. Journal of Vocational Behavior, 83(3), 514-527. [Study 2, Contrast 1: senior mentor versus no mentor]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages","Summary: The study’s objective was to examine the role of informal mentoring on annual compensation, as well as the moderating effects of mentor status. The authors investigated similar research questions in other studies, the profiles of which are available [here]. The authors used statistical methods to analyze self-reported data on earnings, mentoring relationships, and employee characteristics collected through questionnaires administered to the software engineering department of a major aerospace manufacturing firm. The study found no statistically significant differences between the earnings of employees with senior mentors and those with no mentors. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that any estimated effects would be attributable to informal mentoring; however, the study did not find any statistically significant effects.",Mentoring,"Other wages and benefits Mentoring","Adult, Employed",,"United States",2013,https://doi.org/10.1016/j.jvb.2013.08.001,"Employment and Training Review Protocol" "From the shop floor to the kitchen floor: Maternal occupational complexity and children’s reading and math skills","Yetis-Bayraktar, A., Budig, M., & Tomaskovic-Devey, D. (2013). From the shop floor to the kitchen floor: Maternal occupational complexity and children’s reading and math skills. Work and Occupations, 40(1), 37-64.","Women in Science, Technology, Engineering, & Math (STEM)","Study Type: Descriptive Analysis",,,"Summary: The authors sought to determine whether the complexity of a mother’s occupation (or the quality of employment as defined by its task complexity, autonomy, and authority) affected her 6- to 13-year-old children’s math and reading test scores. The study considered both the complexity of the mother’s occupation when her children were ages 6 to 13 as well as the complexity of her occupation in her children’s earliest years (birth to age 3). The authors analyzed data from the 1984–1996 Panel Study of Income Dynamics and its 1997 Child Development Supplement to determine the relationship between maternal occupational complexity and children’s scores on the Revised Woodcock-Johnson Test of Achievement. The analysis first accounted for traits associated with mothers’ employment status, that is, whether the mother was employed. Assuming the mother was employed, the authors then assessed the association between the complexity of the mother’s occupation and her children’s performance on the reading and math sections of the test. The study found that a one-point increase on the maternal white collar occupational complexity scale was associated with a 2.49-point increase in children’s test scores, whereas a one-point increase on the blue collar occupational complexity scale corresponded to a 1.01-point gain in test scores. Maternal occupational complexity during the child’s first three years also was associated with increased test scores later in life; a one-point gain in white collar occupational complexity in the first three years of the child’s life was associated with a 3.7-point increase in test scores when the child was ages 6 to 13.",,"Other employment and reemployment","Employed, Female, Parent",,"United States",2013,,"Women in Science, Technology, Engineering, & Math (STEM) Review Protocol" "Can post-employment services combined with financial incentives improve employment retention for welfare recipients?","Dorsett, R., Hendra, R., Robins, P. K., & Williams, S. (2013). Can post-employment services combined with financial incentives improve employment retention for welfare recipients? Evidence from the Texas Employment Retention and Advancement evaluation. (No. 1413-13). Madison, WI: Institute for Research on Poverty.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Texas Employment Retention and Advancement (ERA) program on employment entry and retention in two sites: Corpus Christi and Fort Worth. The authors conducted a non-experimental study using data originally collected for a randomized controlled trial. They analyzed survey data and state Unemployment Insurance records to estimate the effects of the ERA program on employment entry and retention. The study found a positive and statistically significant relationship between participation in the ERA program and the likelihood of transitioning into employment and between ERA and employment retention at the Corpus Christi site. The quality of causal evidence presented in this report is low because the authors did not account for differences in wages or employment between the ERA and comparison groups before the beginning of the ERA program. This means we are not confident that the estimated effects are attributable to the Texas ERA program; other factors are likely to have contributed.","Texas Employment Retention and Advancement (ERA)","Job search assistance and supportive services Other wages and benefits","Unemployed, Low income, Female",,"United States",2013,http://www.niesr.ac.uk/sites/default/files/publications/dp409_0.pdf,"Review Protocol" "Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females","Dougherty, T. W., Dreher, G. F., Arunachalam, V., & Wilbanks, J. E. (2013). Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females. Journal of Vocational Behavior, 83(3), 514-527. [Study 1, Contrast 2: other mentor versus no mentor]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages","Summary: The study’s objective was to examine the role of informal mentoring on annual compensation, as well as the moderating effects of mentor status and gender. The authors investigated similar research questions in other studies, the profiles of which are available [here]. The authors used statistical methods to analyze self-reported data on earnings, mentoring relationships, and employee characteristics collected through questionnaires administered to graduates of business programs at two large state universities. The study found no statistically significant difference in compensation between study participants with non-senior mentors and those with no mentors. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that any estimated effects would be attributable to informal mentoring; however, the study did not find any statistically significant effects.",Mentoring,"Other wages and benefits Mentoring","Adult, Employed",,"United States",2013,https://doi.org/10.1016/j.jvb.2013.08.001,"Employment and Training Review Protocol" "Do stronger age discrimination laws make Social Security reforms more effective?","Neumark, D., & Song, J. (2013). Do stronger age discrimination laws make Social Security reforms more effective? Journal of Public Economics, 108, 1-16.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Mixed impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study examined whether the strength of state age discrimination protections affected rates of Social Security benefit claiming and employment outcomes of older workers. The study used a nonexperimental design and the data on Social Security claiming and employment outcomes from the Health and Retirement Study (HRS) to estimate the impacts. The study found that among older men affected by the 1983 Social Security reform, age discrimination protections were associated with increases in employment rates between age 62 and the full retirement age and delayed benefit claiming. The quality of casual evidence presented in this report is moderate because the study was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to state age discrimination laws, but other factors might also have contributed.","The 1983 Social Security Reform","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2013,https://ideas.repec.org/a/eee/pubeco/v108y2013icp1-16.html,"Employment and Training Review Protocol" "Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms","Andersson, F., Holzer, H. J., Lane, J. I., Rosenblum, D., & Smith, J. (2013). Does federally-funded job training work? Nonexperimental estimates of WIA training impacts using longitudinal data on workers and firms (Discussion paper no. 7621). Bonn, Germany: IZA. [Dislocated Worker Program ONLY]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) dislocated worker program’s training services on the employment and earnings of dislocated workers. The authors investigated similar research questions in another study. The other study examined the effects of WIA’s adult program’s training services on the employment and earnings of low-income adults. The authors used a regression model with inverse propensity weights to compare the earnings and employment outcomes of WIA-registered dislocated workers who received training services to the outcomes of those who were registered in WIA, but did not receive training services. The study found that employment was initially significantly lower for the treatment group than for the comparison group in the quarters after WIA registration (Quarters 1 and 2 in State A and Quarters 1–4 in State B), but later became and stayed significantly higher in general in Quarters 6 through 12 in both states. The treatment group earned significantly less than the comparison group in the three years after WIA registration. People in the treatment group earned an average of $5,567 and $5,227 less than those in the comparison group in State A and State B, respectively. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the training services received from the WIA dislocated workers program, but other factors might also have contributed.","the Workforce Investment Act (WIA) Dislocated Worker Program","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Dislocated or displaced worker",,"United States",2013,https://www.nber.org/papers/w19446.pdf,"Employment and Training Review Protocol" "Is Internet job search still ineffective?","Kuhn, P., & Mansour, H. (2013). Is Internet job search still ineffective? The Economic Journal, 124 (December), 1213-1233.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts Employment","Summary: The study’s objective was to determine whether the finding from a previous study that Internet job searches increased unemployment duration was upheld using more recent data. Specifically, the authors compared the impact of Internet job searches on unemployment duration using survey data from 2005–2008 to the estimated impact from a prior study on Internet job searches that used 1998–2001 survey data. The study used a nonexperimental design to compare the impacts of Internet job searches on the duration of unemployment at two different time periods. For the first time period, the authors analyzed data from the United States Current Population Survey (CPS) from 1998 to 2001. For the second time period, the authors used data from the National Longitudinal Survey of Youth (NLSY) 1997 sample, covering 2005 to 2008. The study found that in the earlier period (1998–2001) unemployed adults who used the Internet for any job search activities were unemployed 22 percent longer than those who did not use the Internet, whereas in the later period (2005–2008) young adults who used the Internet for job search activities were reemployed 25 percent faster than comparable workers who did not use the Internet as part of their job search strategy. Thus, the study found that the effect of using the Internet as part of a job search strategy on unemployment durations reversed in the 10 years covered by the two studies. The quality of causal evidence presented in this report is low because the authors did not account for other factors that might have affected unemployment duration. This means we are not confident that the estimated effects are attributable to the use of Internet search activities; other factors might have contributed.",,"Job search assistance and supportive services","Adult, Unemployed",,"United States",2013,,"Review Protocol" "Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females","Dougherty, T. W., Dreher, G. F., Arunachalam, V., & Wilbanks, J. E. (2013). Mentor status, occupational context, and protégé career outcomes: Differential returns for males and females. Journal of Vocational Behavior, 83(3), 514-527. [Study 1, Contrast 1: senior male mentor versus no mentor]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the role of informal mentoring on annual compensation, as well as the moderating effects of mentor status and gender. The authors investigated similar research questions in other studies, the profiles of which are available [here]. The authors used statistical methods to analyze self-reported data on earnings, mentoring relationships, and employee characteristics collected through questionnaires administered to graduates of business programs at two large state universities. The study found that employees with senior mentors who were male earned more compensation than those with no mentors. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to informal mentoring; other factors are likely to have contributed.",Mentoring,"Other wages and benefits Mentoring","Adult, Employed",,"United States",2013,https://doi.org/10.1016/j.jvb.2013.08.001,"Employment and Training Review Protocol" "Case study of East Mississippi Community College's plan to assist the Sara Lee Corporation employees due to plant closing (Unpublished doctoral dissertation)","Hanson, B. J. (2012). Case study of East Mississippi Community College's plan to assist the Sara Lee Corporation employees due to plant closing (Unpublished doctoral dissertation). Mississippi State University.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study's objective was to examine the impact of Eastern Mississippi Community College's (EMCC) workforce services training on earnings and employment. The study used a nonexperimental design to compare the outcomes of individuals receiving manufacturing training, non-manufacturing training, or no training. Using data from the Statewide Longitudinal Integrated Workforce Management System, the author conducted statistical models to examine the differences between groups. The study found a positive statistically significant relationship between manufacturing training and employment. The quality of causal evidence presented in this report is low because the author did not include sufficient control variables. This means we are not confident that the estimated effects are attributable to EMCC's workforce services training; other factors are likely to have contributed.","Eastern Mississippi Community College's Workforce Services Training","Employment and Training Services Training and Education Basic skills Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Low-skilled",,"United States",2012,https://ir.library.msstate.edu/handle/11668/20149,"Community College Review Protocol" "What will my account really be worth? An experiment on exponential growth bias and retirement saving","Goda, G.S., Manchester, C.F., & Sojourner, A. (2012). What will my account really be worth? An experiment on exponential growth bias and retirement saving. National Bureau of Economic Research working paper 17927. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine how general or personalized information on projected retirement savings account balances and annual retirement income affects employees’ retirement savings decisions. A group of university employees were randomly assigned to one of four conditions: Participants in the planning treatment group received general retirement savings information, including enrollment instructions for the university’s voluntary retirement plan. Participants in the balance treatment group received all planning treatment materials plus personalized projections of retirement fund balances and access to an online customization tool. Participants in the income treatment group received all planning and balance treatment materials plus a personalized projection of income during retirement. Participants in the control group received no intervention and were in departments with only control group members. The study analyzed retirement-fund contribution data from the university’s Office of Human Resources using fixed-effects regressions. The study also collected and analyzed additional survey data regarding financial literacy, tolerance for delayed gratification, tendency to procrastinate, and beliefs about saving. Participants in the income treatment group contributed $85.42 more on average, annually, to their retirement savings accounts than did members of the control group. Both this group and those in the balance treatment group exhibited statistically significant increases in the probability of changing their contribution amounts of 1.2 and 1.4 percentage points, respectively. The quality of causal evidence provided in this study is high. This means that we are confident that the estimated effects are attributable to the different levels of the intervention and not to other factors.","Providing General Information and Personalized Savings and Income Projections","Employer provided retirement benefits",Employed,,"United States",2012,http://www.nber.org/papers/w17927,"Behavioral Finance: Retirement Review Protocol" "Randomized government safety inspections reduce worker injuries with no detectable job loss","Levine, D., Toffel, M., & Johnson, M. (2012). Randomized government safety inspections reduce worker injuries with no detectable job loss. Science, 336(6083), 907-911.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Health and safety-Mod/high-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of random OSHA inspections on firms’ injury-related outcomes and performance. To do this, the authors used a regression model to compare changes over time in firms randomly chosen to receive inspections to changes in uninspected firms. The study found that the number of injuries and the cost of injuries declined after OSHA inspected a firm. In addition, the study found no evidence that the firms OSHA inspected had lower employment, sales, or credit ratings or were less likely to continue to operate than firms OSHA did not inspect. The quality of causal evidence presented in this study is moderate because the study used a well-conducted nonexperimental design. This means we have confidence that the estimated effects are attributable at least in part to OSHA inspections. However, other factors not accounted for in the study could also have contributed to the estimated effects.","OSHA Enforcement Activities","Inspections Workers compensation Workplace injury",Employed,,"United States",2012,,"OSHA Enforcement Review Protocol" "Does occupational training by the trade adjustment assistance program really help reemployment? Success measured as occupation matching.","Park, J. (2012). Does occupational training by the trade adjustment assistance program really help reemployment? Success measured as occupation matching. Review of International Economics, 20(5), 999-1016.","Apprenticeship and Work-Based Training, Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts Employment","Summary: The study’s objective was to examine the impact of training funded by Trade Adjustment Assistance (TAA) on dislocated workers’ employment. The author used a nonexperimental approach to compare the reemployment rate among TAA beneficiaries who completed a TAA-funded training program with that of those who did not complete training, and compared the reemployment rate among those who participated in each type of TAA-funded training with that of those who did not participate in any training, after adjusting for workers’ characteristics. The study found that, among TAA beneficiaries, completion of a training program was associated with higher reemployment rates. The study also found that participation in TAA-funded customized training was associated with lower reemployment rates, while participation in occupational training or in on-the-job training was associated with higher reemployment rates compared with not participating in training. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the program. This means we are not confident that the estimated effects are attributable to TAA-funded training; other factors are likely to have contributed.","Trade Adjustment Assistance (TAA)-Funded Training","Community college education and other classroom training Work based and other occupational training","Dislocated or displaced worker",,"United States",2012,,"Apprenticeship and Work-Based Training, Job Search Assistance Review Protocol" "The efficacy of supported employment for middle-aged and older people with schizophrenia.","Twamley, E. W., Vella, L., Burton, C. Z., Becker, D. R., Bell, M. D., & Jeste, D. V. (2012). The efficacy of supported employment for middle-aged and older people with schizophrenia. Schizophrenia Research, 135(1-3), 100-104.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of Individual Placement and Support (IPS) on competitive employment for older people with schizophrenia The study was a randomized controlled trial (RCT) conducted at a community mental health clinic and used weekly work logs cross-referenced with pay stubs to measure employment outcomes over a 12-month period. The study found that IPS was positively associated with employment and earnings. The quality of causal evidence presented in this report is low because the RCT has a confounding factor. This means we are not confident that the estimated effects are attributable to the IPS program; other factors are likely to have contributed.","Individual Placement and Support and Vocational Rehabilitation Program","Health Individuals facing barriers to employment Job search assistance and supportive services","Adult, Older worker, Disability",,"United States",2012,https://psycnet.apa.org/record/2011-30184-001,"Employment and Training Review Protocol" "Investing for retirement: The moderating effect of fund assortment size on the 1/n heuristic","Morrin, M., Inman, J. J., Broniarczyk, S. M., Nenkov, G., & Reuter, J. (2012). Investing for retirement: The moderating effect of fund assortment size on the 1/n heuristic. Fox School of Business Research Paper No. 14-009, 1–38.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: Experiments using U.S. households The study’s objective was to examine whether increases in the number of funds offered in a retirement plan caused investors to allocate their contributions more evenly across all available investment options. The authors administered one survey to a nationally representative sample of U.S. households and two surveys to people from consumer panels. The surveys contained information on hypothetical savings plans that were manipulated to include either a small or large fund assortment size. Participants were instructed to choose the plans in which they would invest and indicate the percentage they would invest in each of the chosen funds. The study found that participants were more likely to allocate hypothetical investments across all available funds when offered a small versus large assortment size. The quality of the causal evidence presented in this portion of the study is moderate because, although the authors controlled for observable characteristics of samples members and there was no evidence that survey versions were systematically assigned to sample members, the Clearinghouse for Labor Evaluation and Research (CLEAR) could not confirm that survey versions were randomly assigned. Analysis of employees’ data  The study’s objective was to determine whether the number of fund options offered by a 401(k) plan influenced the investment behavior of plan members. The authors compared investment behaviors among employees offered 10 or 19 fund options for investment, controlling for employees’ characteristics. The analysis used administrative data on contributions to the defined contribution plan of the Oregon University System. The study used an interrupted time series (ITS) design that analyzed changes in outcomes before and after an increase in the number of available investment options. The analysis demonstrated that offering a larger fund assortment size was associated with a statistically insignificant decrease in the tendency of employees to invest in all available funds and spread invested dollars evenly among chosen alternatives. The quality of causal evidence presented in this study is low. This means that we are not confident that the estimated effects are solely attributable to fund assortment size; other factors are likely to have contributed.","Informational Interventions for Households and Workers","Employer provided retirement benefits",Employed,,"United States",2012,,"Behavioral Finance: Retirement Review Protocol" "Impact of the Reemployment and Eligibility Assessment (REA) Initiative in Nevada","Michaelides, M., Poe-Yamagata, E., Benus, J., & Tirumalasetti, D. (2012). Impact of the Reemployment and Eligibility Assessment (REA) Initiative in Nevada. Columbia, MD: IMPAQ International, LLC.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to estimate the impact of the Reemployment and Eligibility Assessment (REA) Initiative in Nevada on Unemployment Insurance (UI) benefits receipt, employment, and earnings after six quarters.  The study was a randomized controlled trial. Of some 33,000 first-time UI claimants eligible for the study, about 5,100 were randomly assigned to the treatment group, which had to take part in REA and reemployment services (RES) to retain UI eligibility, and the rest were randomly assigned to the control group, which was not required to participate in such services. The authors compared state administrative records on UI payments and covered employment of the two groups to estimate the program’s effectiveness. The study found that, in the first six calendar quarters following the initial claim, REA services substantially reduced the average duration and amount of UI benefits receipt, increased employment rates, and increased earnings. The quality of causal evidence presented in this study is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA services, and not other factors.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2012,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_08_REA_Nevada_Foll…,"Reemployment Review Protocol" "Pre- and post- wage differences of trade adjustment assistance job training participants in Arkansas. (Doctoral dissertation).","Gordon, K.H. (2012). Pre- and post- wage differences of trade adjustment assistance job training participants in Arkansas. (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I. (1266388644).",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Unfavorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the Trade Adjustment Assistance (TAA) Program on the wages of dislocated workers. The author used a nonexperimental method to compare the earnings outcomes of participants before and after they participated in the TAA program. The author also compared the change in earnings between those who completed the TAA-funded training and those who started but did not complete TAA-funded training The study found that participants had significantly lower wages in the three quarters after TAA program participation compared with the three quarters before program participation. The study also found that the change in wages differed by education level and employment tenure. The quality of causal evidence presented in this report is low because the author did not account for pre-intervention trends in earnings or account for differences between the treatment and comparison groups. This means we are not confident that the estimated effects are attributable to TAA program; other factors are likely to have contributed.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker",,"United States",2012,http://scholarworks.uark.edu/cgi/viewcontent.cgi?article=1631&context=etd,"Review Protocol" "Income tax and older American workers’ job transition into self-employment.","Choi, S. (2012). Income tax and older American workers’ job transition into self-employment. Korea and the World Economy, 13(2), 189-214.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment","Summary: The study examined the effect of a progressive income tax on the likelihood of transitioning to self-employment among older workers. The study used a nonexperimental approach (the exponential hazard model) that compared the likelihood of transitioning from a wage job to self-employment based on the marginal taxes a worker would expect to pay. The author used the data from the Health and Retirement Study in 1992 matched to the Social Security Administration Master Earnings File from 1980 to 1991, which contains W-2 records of earnings from both wage work and self-employment. The study found that a more progressive tax (that is, a tax schedule that increases sharply toward the higher end) or higher marginal tax rates reduces the likelihood of entering self-employment. The quality of causal evidence presented in this report is low because the author did not ensure that the groups with higher or lower marginal tax rates were similar before a change in the tax schedule. This means we are not confident that the estimated effects are attributable to changes in marginal taxes; other factors are likely to have contributed.","Progressive Income Tax","Other employment and reemployment","Adult, Older worker, Employed",,"United States",2012,,"Employment and Training Review Protocol" "Employment growth from public support of innovation in small firms","Link, A., & Scott, J. (2012). Employment growth from public support of innovation in small firms. Economics of Innovation and New Technology, 21(7), 655-678.","Entrepreneurship and Self-Employment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the effects of the Small Business Innovation Research (SBIR) program, a competitive program providing small businesses with funding for technological innovations, on employment growth of SBIR-recipient firms. The study used regression analysis to compare firms’ actual levels of employment in 2005 with the levels of employment predicted by their characteristics before receiving the Phase II award. The study found no statistically significant relationships between SBIR funding and firms’ employment. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to SBIR funding; other factors are likely to have contributed.","the SBIR Program","Other training and education",Self-employed,"Small business","United States",2012,,"Employment and Training Review Protocol" "Returning to work after prison: Final results from the Transitional Jobs Reentry Demonstration","Jacobs, E. (2012). Returning to work after prison: Final results from the Transitional Jobs Reentry Demonstration. New York: MDRC.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Transitional Jobs Reentry Demonstration (TJRD) on ex-offenders’ employment and wages. The author used a randomized controlled trial to assign former prisoners to the transitional jobs program or to a group that was offered job search assistance and compared the two groups’ employment and earnings outcomes two years after random assignment, controlling for participants’ characteristics before random assignment. The study found that members of the transitional jobs group were significantly more likely (by 29 percentage points) than control group members to be employed at any point during the two years after random assignment. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the TJRD, and not to other factors.","the Transitional Jobs Reentry Demonstration (TJRD)","Job search assistance and supportive services Other employment and reemployment Reentry","Justice-involved, Unemployed, Male",,"United States",2012,http://www.mdrc.org/sites/default/files/full_626.pdf,"Employment and Training Review Protocol" "Thrift Savings Plan. (2012). Participant behavior and demographics: Analysis of 2008–2012.","Thrift Savings Plan. (2012). Participant behavior and demographics: Analysis of 2008–2012.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Mixed impacts Employer benefits receipt","Summary: The study’s objective was to determine whether changes in the design of the Thrift Savings Plan (TSP) led to changes in federal employees’ participation in and contributions to the Federal Employee Retirement System (FERS). In 2009 and 2010, TSP instituted two major changes, a decrease in the waiting period for eligibility to receive employer matching contributions and automatic enrollment in a Government Securities Investment Fund (G Fund). This study used an interrupted time series design to compare employees’ investment behaviors before and after the TSP design changes. It used data from the TSP record-keeping system and the Office of Personnel Management (OPM). Across the TSP participants analyzed, participation in FERS increased by about 4 percentage points from 2009 to 2012, whereas the average deferral rate decreased slightly from the 2008 level. TSP’s design changes were not associated with how often employees monitored their FERS investments. The quality of causal evidence presented in this study is low, meaning that we are not confident that the estimated effects are attributable to TSP design changes. Other factors are likely to have contributed.","Immediate Contributions and Automatic Enrollment in the Thrift Savings Plan","Employer provided retirement benefits",Employed,,"United States",2012,,"Behavioral Finance: Retirement Review Protocol" "A new estimate of the impact of OSHA inspections on manufacturing injury rates, 1998-2005","Haviland, A., Burns, R., Gray, W., Ruder, T., & Mendeloff, J. (2012). A new estimate of the impact of OSHA inspections on manufacturing injury rates, 1998-2005. American Journal of Industrial Medicine, 55(11), 964-975.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine the effect of OSHA inspections on injuries in single-facility manufacturing firms in Pennsylvania between 1998 and 2005. The study used a regression model to compare the changes in workplace injuries of firms that had received particular types of inspections to those that had not received those types of inspections. The study found that inspections with penalties were associated with a statistically significantly decrease in injuries by an average of 19 to 24 percent annually in the two years after the inspection. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in workplace injuries between firms that received particular types of OSHA inspections and firms that did not receive those types of inspections are attributable to the inspections.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,Manufacturing,"United States",2012,,"OSHA Enforcement Review Protocol" "Estimated impacts for participants in the Trade Adjustment Assistance (TAA) program under the 2002 amendments.","Schochet, P.Z., D’Amico, R., Berk, J., Dolfin, S., & Wozny, N. (2012). Estimated impacts for participants in the Trade Adjustment Assistance (TAA) program under the 2002 amendments. Princeton, NJ: Mathematica Policy Research.",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Unfavorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Unfavorable impacts Employment Public benefits receipt-Mod/high-Unfavorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Trade Adjustment Assistance (TAA) Program on education and training, employment, earnings, and public benefit receipt. The study used a nonexperimental method to match those who took part in the TAA Program to a similar group of nonparticipants. The authors used data from telephone surveys and administrative records to compare education, employment, earnings, and public benefits receipt outcomes across the groups. The study found that education and training programs completion and educational attainment were higher among TAA participants than similar nonparticipants. Employment, earnings, and receipt of cash assistance were lower for TAA participants than for similar nonparticipants, and a larger percentage of TAA participants than nonparticipants received Unemployment Insurance (UI) benefits and food stamps. The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to TAA, but other factors might also have contributed.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other training and education","Dislocated or displaced worker",,"United States",2012,https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&…,"Review Protocol" "Small cues change savings choices","Choi, J., Haisley, E., Kurkoski, J., & Massey, C. (2012). Small cues change savings choices. National Bureau of Economic Research working paper 17843. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to gauge the effect of savings cues on 401(k) contribution rates by exposing randomly selected groups of employees to different hypothetical savings rates or amounts. The authors analyzed the relationship between exposure to savings rate cues and savings choices among employees at an unnamed U.S. technology company who contributed less than the maximum yearly amount to their retirement savings plans prior to the intervention. Overall, the authors found that participants adjusted their 401(k) contribution rates in response to cues. Those who were cued with the maximum contribution amount were 5.7 percent more likely than those who received no cue to contribute at maximal levels. The quality of causal evidence provided in this study is low because it is a randomized controlled trial with unknown attrition and unknown equivalence of the analytic groups. This means we are not confident the estimated effects are attributable to savings cues; other factors are likely to have contributed.","Savings Cue","Employer provided retirement benefits",Employed,,"United States",2012,http://www.nber.org/papers/w17843,"Behavioral Finance: Retirement Review Protocol" "The impact of regulatory enforcement and consultation visits on workers’ compensation claims incidence rates and costs, 1999–2008","Foley, M., Fan, J., Rauser, E., & Silverstein, B. (2012). The impact of regulatory enforcement and consultation visits on workers’ compensation claims incidence rates and costs, 1999–2008. American Journal of Industrial Medicine, 55(11), 976-990.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine the impact of inspections and consultations by Washington State’s Department of Occupational Safety and Health (DOSH) on workers’ compensation claim rates and their associated costs. The study used regression models to compare changes in the claim rates and claim costs across years for firms that had a DOSH inspection or consultation with those that did not. The study found that DOSH activities were associated with statistically significant reductions in workers’ compensation claim rates. The study also found reductions in claims’ costs in firms with DOSH activities, but this was not statistically significant. The quality of the causal evidence presented in this study is low. This means we do not have confidence that the effects estimated in this study are attributable to DOSH activities.","OSHA Enforcement Activities","Inspections Other health and safety Workers compensation Workplace injury",Employed,,"United States",2012,,"OSHA Enforcement Review Protocol" "Does trade adjustment assistance make a difference?","Reynolds, K.M., & Palatucci, J.S. (2012). Does trade adjustment assistance make a difference? Contemporary Economic Policy, 30(1), 43-59. doi:10.1111/j.1465-7287.2010.00247.x",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Trade Adjustment Assistance (TAA) Program on the employment and earnings of displaced workers in the manufacturing sector from 2003 to 2005. The study uses a nonexperimental design to compare the outcomes of TAA recipients with a comparison group of nonrecipients who were also displaced from manufacturing jobs during the same period as the treatment group. The study found no statistically significant relationship between TAA participation and employment or earnings when comparing TAA recipients with displaced workers who did not participate in TAA. However, when comparing the TAA recipients who received training with recipients who did not receive training, the study found a significant relationship between training and employment and earnings. The quality of causal evidence provided in this study is low because the authors did not sufficiently account for potential differences between the study groups. This means we are not confident that estimated effects are attributable to the TAA program; other factors are likely to have contributed.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other training and education","Dislocated or displaced worker",,"United States",2012,,"Review Protocol" "Does trade adjustment assistance make a difference?","Reynolds, K. M., & Palatucci, J. S. (2012). Does trade adjustment assistance make a difference? Contemporary Economic Policy, 30(1), 43-59. [TAA with training versus TAA without training]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of Trade Adjustment Assistance (TAA) on the employment and earnings of participants displaced from jobs in the manufacturing sector who did and did not receive training services through TAA. The authors investigated similar research questions in another study examining the impact of TAA services relative to no TAA services, the profile of which is available [here]. The study uses a nonexperimental design to compare the outcomes of TAA participants who received job training and other program services with outcomes of a comparison group of participants who obtained a training waiver but received other forms of program assistance. Data for program participants were based on U.S. Department of Labor Trade Act Participant reports. The study found statistically significant relationships between receiving job training through the TAA program and employment and earnings when compared with participants who obtained a training waiver but received other forms of program assistance. The quality of causal evidence provided in this study is low because the study uses a nonexperimental design, and the authors did not demonstrate that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to TAA; other factors are likely to have contributed.","Trade Adjustment Assistance (TAA)-Funded Training","Adult and Dislocated Worker programs Job search assistance and supportive services Other training and education","Dislocated or displaced worker",,"United States",2012,http://faculty.smu.edu/Millimet/classes/eco7377/papers/reynolds%20palatucci%202…,"Employment and Training Review Protocol" "Tax elasticity of labor earnings for older individuals","Alpert, A. & Powell, D. (2012). Tax elasticity of labor earnings for older individuals. (Report no. 2012-272). Ann Arbor, MI: University of Michigan Retirement Research Center, University of Michigan.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment","Summary: The study examined the impact of tax changes on people’s employment, retirement, and labor income outcomes. The study uses a nonexperimental approach and the data from the Health and Retirement Study to estimate the effect. The study found positive statistically significant relationships between increases in after-tax labor income and the employment of men and women as well as the delay in retirement for men. The study also found a positive statistically significant relationship between increases in the marginal net-of-tax rate and earnings for women. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to tax changes from the Economic Growth and Tax Relief Reconciliation Act (EGTRRA), the Jobs and Growth Tax Relief Reconciliation Act (JGTRRA), and their marriage tax penalty relief provision, but other factors might also have contributed.","Tax Changes","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2012,https://deepblue.lib.umich.edu/bitstream/handle/2027.42/94533/wp272.pdf?sequenc…,"Employment and Training Review Protocol" "Notes on behavioral economics and labor market policy","Babcock, L., Congdon, W., Katz, L., & Mullainathan, S. (2012). Notes on behavioral economics and labor market policy. IZA Journal of Labor Policy, 1(1), 1-14.","Behavioral Insights","Study Type: Descriptive Analysis",,,"Summary: The article’s objective was to assess how behavioral economics can inform labor policy reforms to increase policy efficiency. The authors used behavioral economic theory to explain barriers to successful policies related to unemployment insurance, job search assistance, and job training. They also proposed policy solutions to overcome these barriers. For unemployment insurance, the authors recommended implementing wage loss insurance that subsidizes reemployment wages to counteract individuals’ tendencies to set high reservation wages when seeking employment, which slows their return to work. They also recommended providing small, immediate, and high frequency reminders and incentives to search for work. To increase participation in job search assistance, the authors recommended policies that automatically put unemployed individuals into services and/or to make job search assistance more user-friendly and personalized. Similarly, to increase participation in job training programs, the authors recommended streamlining or simplifying training offerings and providing guidance to reduce the burden on the prospective users to select and start a program. Simplifying the decision-making process by creating a competitive training market could also be beneficial to prospective users.",,"Job search assistance and supportive services Unemployment Insurance Other training and education","Adult, Unemployed",,"United States",2012,,"Behavioral Insights Review Protocol" "Impact of the Reemployment and Eligibility Assessment Initiative","Poe-Yamagata, E., Benus, J., Bill, N., Carrington, H., Michaelides, M., & Shen, T. (2011). Impact of the Reemployment and Eligibility Assessment Initiative. Columbia, MD: IMPAQ International. [Florida sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) Initiative in Florida, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on UI benefits receipt, employment, and earnings. The study randomly assigned 80,531 eligible UI claimants to either the treatment group or the control group. Those in the treatment group were required to participate in REA services to retain their UI eligibility. Data for the analysis came from UI administrative and wage records.  The study found statistically significant reductions in weeks of UI benefits receipt, total amount of UI benefits received, and the probability of benefits exhaustion for the REA treatment group. In addition, participants in REA had a statistically significantly higher probability of employment and earnings over the four follow-up quarters. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA initiative, and not to other factors.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2011,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_08_Impact_of_the_R…,"Reemployment Review Protocol" "Labor market rigidities and the employment behavior of older workers.","Blau, D., & Shvydko, T. (2011). Labor market rigidities and the employment behavior of older workers. Industrial and Labor Relations Review, 64(3), 464-484.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of flexibility in hours of work on the likelihood that older workers separate from an employer. The study used a statistical model to compare the job exit behavior of older and younger workers in firms with different shares of women younger than age 30. The analysis used monthly data on workers ages 45 to 69 in the Survey of Income and Program Participation from 1990 to 2001 matched to employers in the Longitudinal Employment and Household Dynamics data. The study found that workers in their sixties who work for employers with a higher share of women younger than age 30 are less likely to experience job separations compared with workers in their late 40s to mid-50s. The quality of causal evidence presented in this report is low because the groups being compared might not be similar before leaving their employer. This means we are not confident that the estimated effects are attributable to employment flexibility, other factors are likely to have contributed.","Employment flexibility","Other employer services","Adult, Older worker",,"United States",2011,https://www.jstor.org/stable/41149475?seq=1#page_scan_tab_contents,"Employment and Training Review Protocol" "Earnings progression among workforce development participants: Evidence from Washington State.","Chrisinger, C.K. (2011). Earnings progression among workforce development participants: Evidence from Washington State. Washington, DC: U.S. Department of Labor, Employment and Training Administration.","Job Search Assistance, Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Adult Program on the earnings of low-income adults ages 18 to 64 in Washington State. The author assigned workers to the treatment group if they received employment-related services through the WIA Adult Program from January 2002 to June 2008. Workers who participated in the Labor Exchange program during this time comprised the comparison group. The study found no statistically significant relationships between receiving services through the WIA Adult Program and quarterly earnings. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before program participation. This means we would not be confident that the estimated effects would be attributable to the WIA Adult Program; however, the study did not find statistically significant effects.","the Workforce Investment Act (WIA) Adult Program","Adult and Dislocated Worker programs Job search assistance and supportive services Labor Exchange Other employment and reemployment Other training and education Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Unemployed, Low income",,"United States",2011,https://wdr.doleta.gov/research/FullText_Documents/ETAOP_2011-11.pdf,"Job Search Assistance, Low-Income Adults Review Protocol" "Would people behave differently if they better understood Social Security? Evidence from a field experiment","Liebman, J., & Luttmer, E. (2011). Would people behave differently if they better understood Social Security? Evidence from a field experiment. National Bureau of Economic Research working paper no. 17287. Cambridge, MA: NBER.","Behavioral Finance: Retirement, Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of providing information about Social Security rules and benefits on labor force participation, knowledge of Social Security, and claiming of Social Security benefits. Workers nearing retirement age were randomly assigned into either the treatment group, which was sent an informational brochure about Social Security and invited to a 15-minute web tutorial on Social Security provisions, or to the control group, which was not offered this information but could seek other publicly available information about Social Security. The authors measured outcomes through a follow-up survey conducted 13 months after random assignment. The study found that, on average, members of the treatment group were more likely to work for pay in the last month and were more likely to be aware that Social Security benefits were based on the number of years with the highest earnings, compared with the control group. However, there were no statistically significant differences between the two groups on any other outcomes related to earnings and employment, Social Security benefit receipt, or understanding Social Security rules. The quality of causal evidence presented in this report is moderate because it was based on a well-conducted randomized controlled trial with high attrition and adequate control variables. This means we are somewhat confident that the estimated effects are attributable to providing information about Social Security provisions, but other factors might also have contributed.",,,Employed,,,2011,http://www.nber.org/papers/w17287.pdf,"Behavioral Finance: Retirement, Behavioral Insights Review Protocol" "Adolescent perspectives of the ecological impact of a summer youth employment program.","Grant, N. (2011). Adolescent perspectives of the ecological impact of a summer youth employment program. Doctoral dissertation. Wichita, KS: Wichita State University. Available at http://soar.wichita.edu/handle/10057/5056.","Opportunities for Youth","Study Type: Descriptive Analysis",,,"Summary: Using personal stories of youth who participated in the Area IV Kansas Summer Youth Employment Program (SYEP), the author sought to describe the ecosystems of urban youth, their relationships with adults, and the impacts of their participation in SYEP. SYEP was a summer program funded as part of the American Recovery and Reinvestment Act of 2009. It provided disadvantaged youth with summer employment and job skills training over two summers. The author interviewed 10 participating youth and analyzed their personal stories using ecological systems theory and a social capital framework. The author also analyzed documents containing the youths’ demographic and employment records. The author concluded that programs such as SYEP helped disadvantaged youth to make networking connections, learn ways of interacting in the workplace, and open doors to job opportunities that historically have not been available to those with lower socioeconomic status.","Summer Youth Employment Programs (Area IV Kansas)","Basic skills Work based and other occupational training Youth programs","Youth, Disconnected youth, Low-skilled, Low income",,"United States",2011,http://www.workforce-ks.com/modules/showdocument.aspx?documentid=2206,"Opportunities for Youth Review Protocol" "Long-term findings from an experimental evaluation of three service delivery models.","Perez-Johnson, I., Moore, Q., & Santillano, R. (2011). Long-term findings from an experimental evaluation of three service delivery models. Series: ETAOP 2012-06. Washington, DC: Employment and Training Administration, U.S. Department of Labor.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of individual training account (ITA) programs on employment, earnings, education and training, and benefit receipt. The authors conducted a randomized controlled trial. They used Unemployment Insurance (UI) records of the full sample and a follow-up survey conducted on average seven years after random assignment for a random subsample of the full sample (4,800 of 7,920) to determine education and training, employment, and earnings outcomes. The study found that a significantly higher percentage of the maximum customer choice group (59 percent) had earned a certificate or degree from a training program within 3 years of random assignment compared with the guided customer choice group (53 percent). The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the ITA models, and not to other factors.","The Individual Training Accounts (ITA) models","Other employment and reemployment Other training and education Unemployment Insurance","Low income, Dislocated or displaced worker",,"United States",2011,https://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_06.pdf,"Employment and Training Review Protocol" "Impact of the Reemployment and Eligibility Assessment Initiative","Poe-Yamagata, E., Benus, J., Bill, N., Carrington, H., Michaelides, M., & Shen, T. (2011). Impact of the Reemployment and Eligibility Assessment Initiative. Columbia, MD: IMPAQ International. [Nevada sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) Initiative in Nevada, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on duration and amount of UI benefits receipt. The study was a randomized controlled trial. Of some 33,000 first-time UI claimants eligible for the study, about 5,100 were randomly assigned to the treatment group, which had to take part in REA and reemployment services (RES) to retain UI eligibility, and the rest were randomly assigned to the control group, which was not required to participate in such services. The authors compared state administrative records on UI benefits to estimate the program’s effectiveness. The study found statistically significant reductions in weeks of UI benefits receipt, total amount of UI benefits received, total number of weeks receiving UI benefits, and the probability of benefits exhaustion for the treatment group. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA initiative, and not to other factors.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2011,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_08_Impact_of_the_R…,"Reemployment Review Protocol" "Workforce Investment Act training for older workers: Toward a better understanding of older worker needs during the economic recovery (ETA Occasional Paper 2011-10).","Zhang, T. (2011). Workforce Investment Act training for older workers: Toward a better understanding of older worker needs during the economic recovery (ETA Occasional Paper 2011-10). Washington, DC: U.S. Department of Labor, Employment and Training Administration.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Mixed impacts Employment","Summary: The study examined the impacts of various Workforce Investment Act (WIA) programs and services on the employment rates of older participants who exited these programs. The study used a statistical model (fixed and random effect) to estimate the correlation between WIA programs and services and the Entered Employment Rate one quarter after participants age 55 and older exited the programs. The study used the WIA Standardized Record Data from January 1, 2006, to September 30, 2007, and data on state-level unemployment rates and trends from a Bureau of Labor Statistics data set. The study found that some characteristics of the WIA training programs and services are associated with higher employment rates, while other characteristics are associated with lower employment rates. The quality of causal evidence presented in this report is low because the study did not demonstrate that participants across different training programs were similar before the study and did not account for possible differences in the groups. This means we are not confident that the estimated effects are attributable to the WIA programs and services; other factors are likely to have contributed.","the Workforce Investment Act (WIA) training for older workers","Adult and Dislocated Worker programs Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Adult, Unemployed",,"United States",2011,,"Employment and Training Review Protocol" "Impact of the Reemployment and Eligibility Assessment Initiative","Poe-Yamagata, E., Benus, J., Bill, N., Carrington, H., Michaelides, M., & Shen, T. (2011). Impact of the Reemployment and Eligibility Assessment Initiative. Columbia, MD: IMPAQ International. [Illinois sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) Initiative in Illinois, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on duration and amount of UI benefits receipt. The study was a randomized controlled trial that assigned 3,112 eligible UI claimants to the treatment or control group. Those in the treatment group were required to participate in REA services to retain their UI eligibility. Data for the analysis came from UI administrative records. The study found no statistically significant impacts on any UI benefits receipt outcomes examined. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA initiative, and not to other factors.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed, High-skilled",,"United States",2011,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_08_Impact_of_the_R…,"Reemployment Review Protocol" "$100 bills on the sidewalk: Suboptimal investment in 401(K) plans","Choi, J., Laibson, D., & Madrian, B. (2011). $100 bills on the sidewalk: Suboptimal investment in 401(K) plans. The Review of Economics and Statistics, 93(3), 748-763.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-No impacts Employer benefits receipt","Summary: The study examined the impact of providing information about employer contribution matches and penalty-free withdrawal rules on 401(k) contribution rates. These features created an opportunity for certain employees to increase their retirement savings without decreasing take-home pay. Even though this enabled employees to make a profit at very little cost, many eligible employees did not choose to exploit this opportunity. Employees were randomly assigned to receive a survey embedded with explicit information on the opportunity or a control survey without the information. The authors used survey data linked to administrative data obtained from Hewitt Associates, a large benefits administration and consulting firm. The study found no statistically significant relationship between the treatment condition and future 401(k) contribution rates. The quality of the causal evidence presented in this study is high. This means we are confident that any estimated effects of the intervention would be attributable to the intervention itself, and not some other factor.","the Provision of Information on a Missed ""Free Lunch""","Employer provided retirement benefits",Employed,,"United States",2011,,"Behavioral Finance: Retirement Review Protocol" "Beyond a summer work experience: The Recovery Act 2009 Post-Summer Youth Employment Initiative.","Holcomb, P., Ziegler, J., & Laird, E. (2011). Beyond a summer work experience: The Recovery Act 2009 Post-Summer Youth Employment Initiative. Princeton, NJ: Mathematica Policy Research.","Opportunities for Youth","Study Type: Implementation Analysis",,,"Summary: Funded by the American Recovery and Reinvestment Act of 2009, the Summer Youth Employment Initiative (SYEI) aimed to reverse the steep decline in youth employment during the recession by providing employment opportunities for youth in the summer of 2009. The 2009 Post-SYEI allowed states and local workforce investment areas to provide extended work experiences to older disconnected youth ages 18 to 24 from October 2009 through March 2010. The authors conducted in-depth site visits to eight selected study sites and analyzed the data using qualitative methods. The authors reported that the study sites effectively built upon the implementation of the SYEI by using the same service delivery structure and extending the work experiences of those youth who had already participated in the summer program. The longer time frame for the work experiences was seen as a benefit by participants and employers. Post-SYEI programs also focused on ensuring the alignment between participants’ interests and their work experiences. The authors recommended three considerations for the design of similar policies targeting older out-of-school youth in the future: allowing sufficient time to design such programs, because the needs of older out-of-school youth differ from those of younger youth who are traditionally served by these types of programs; supporting a year-round, not only summer, work experience; and incorporating a transition strategy from the work experience into future opportunities.","Summer Youth Employment Initiative (SYEI)","Basic skills Job search assistance and supportive services Work based and other occupational training Youth programs","Youth, Disconnected youth, Low-skilled, Low income",,"United States",2011,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_201_03.pdf,"Opportunities for Youth Review Protocol" "Boon or bane?: 401(k) loans and loan provisions","Wenger, J., & Weller, C. (2011). Boon or bane?: 401(k) loans and loan provisions. Available at http://ssrn.com/abstract=1941411.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to examine the impact of having a borrowing option in a defined contribution (DC) pension plan on contribution rates and total debt. The authors used data from the United States Federal Reserve’s triennial Survey of Consumer Finance to estimate ordinary least squares (OLS) and instrumental variable (IV) regression models. Both models control for demographic characteristics and financial situation. The authors found that having a borrowing option significantly increased both contribution rates and debt. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the borrowing option in DC pension plans. Other factors are likely to have contributed.","Borrowing Options in Defined Contribution Pension Plans","Employer provided retirement benefits",Employed,,"United States",2011,http://ssrn.com/abstract=1941411,"Behavioral Finance: Retirement Review Protocol" "Evaluation of the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care: Final Report","Courtney, M., Zinn, A., Johnson, H., and Malm, K. (2011). Evaluation of the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care: Final Report. OPRE Report #2011-14. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care (Outreach) on youths’ education, employment, earnings, and public benefit receipt. Using a randomized controlled trial design, the authors estimated the program’s impact by comparing survey responses and college records for youth randomly selected to receive the Outreach program with those of youth randomly assigned to the control group. The study found that the Outreach program significantly increased college enrollment and persistence. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care, and not to other factors.","the Massachusetts Adolescent Outreach Program for Youths in Intensive Foster Care","Job search assistance and supportive services Other training and education Youth programs Behavioral Interventions","Youth, Disconnected youth",,"United States",2011,http://www.acf.hhs.gov/sites/default/files/opre/eval_mass.pdf,"Opportunities for Youth Review Protocol" "Impact of the Reemployment and Eligibility Assessment Initiative","Poe-Yamagata, E., Benus, J., Bill, N., Carrington, H., Michaelides, M., & Shen, T. (2011). Impact of the Reemployment and Eligibility Assessment Initiative. Columbia, MD: IMPAQ International. [Idaho sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) Initiative in Idaho, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on duration and amount of UI benefits receipt. The study randomly assigned 18,156 eligible claimants to either the treatment group, which was eligible for REA services, or the control group, which was not offered REA services. Within the treatment group, claimants were randomly assigned either to receive a letter requiring them to enter work search activities into an online system, or to receive both the letter and an in-person interview. Data for the analysis comes from UI administrative records. The study found statistically significant reductions in weeks of UI benefits receipt, total amount of UI benefits received, total number of weeks receiving UI benefits, and the probability of benefits exhaustion for the treatment group. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA initiative, and not to other factors.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2011,http://wdr.doleta.gov/research/FullText_Documents/ETAOP_2012_08_Impact_of_the_R…,"Reemployment Review Protocol" "Misunderstanding savings growth: Implications for retirement savings behavior","McKenzie, C., & Liersch, M. (2011). Misunderstanding savings growth: Implications for retirement savings behavior. Journal of Marketing Research, 68, S1–S13.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: Experiments Using Undergraduate Students This study used three experiments in a controlled setting to examine undergraduate students’ intuitions of retirement savings growth, understanding of investment concepts, and motivation to invest toward their retirement after entering the workforce.1 The authors administered three surveys consisting of hypothetical situations followed by exercises asking respondents to compute an omitted element of the savings function (for example, the monthly deposit amount or rate of return required to accumulate a given amount of savings at retirement). Each experiment randomly assigned participants to receive or not receive some form of aid (for example, a calculator) or additional information (for example, savings growth charts) to complete the exercises. The first study found that participants tended to severely underestimate projected savings account balances at retirement and overestimate the rates of return and monthly deposit amounts required to ensure such balances. The second study found that when provided with information on exponential growth, participants were more motivated to save for retirement; however, in the third study there was no statistically significant relationship between information on exponential growth and motivation to save for retirement. The quality of the causal evidence presented in this portion of the study is high. This means we are confident that any differences in responses were a result of the various treatment conditions of the three experiments. Analysis of Employees’ Survey Data The study further aimed to determine whether showing employees how much money they would have at retirement—based on current annual contributions, the current account balance, and return rates—can motivate them to increase savings, compared with showing employees current account balances only. The authors analyzed differences in employees’ interest in changing current levels of retirement savings, measured on a seven-point Likert scale, comparing those who were provided only their estimated 401(k) account balance at retirement and those who were shown only their current balance. The analysis demonstrated that showing workers actual account projections (in addition to the current balance) motivated them to save more. The quality of causal evidence presented in this portion of the study is also high. This means we are confident that showing employees their estimated 401(k) account balances at retirement increased the percentage of employees who wanted to save more each month.","Informational Interventions for Undergraduates and Workers","Employer provided retirement benefits","Adult, Employed",,"United States",2011,,"Behavioral Finance: Retirement Review Protocol" "Baltimore County Juvenile Drug Court outcome and cost evaluation","Mackin, J., Lucas, L., Lambarth, C., Herrera, T., Waller, M., Carey, S., & Finigan, M. (2010). Baltimore County Juvenile Drug Court outcome and cost evaluation. Portland, OR: NPC Research.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study examined the impact of participation in the Baltimore County Juvenile Drug Court on drug use and recidivism. The authors used a nonexperimental design, matching youth who participated in the juvenile drug court with a comparison group of similar youth who were eligible to participate in juvenile drug court, but instead participated in the traditional juvenile justice system. The authors used administrative data to compare the mean number of juvenile rearrests over a two-year follow-up period. The study found that drug court participants had significantly fewer juvenile rearrests, both overall and specifically for drug charges, than comparison youth. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Baltimore County Juvenile Drug Court, but other factors might also have contributed.","the Baltimore County Juvenile Drug Court","Substance abuse recovery Youth programs","Youth, Disability, Disconnected youth",,"United States",2010,http://www.ndcrc.org/sites/default/files/baltimore_co_juv_dc_outcome-cost_0110…,"Opportunities for Youth Review Protocol" "Simplification and saving","Beshears, J., Choi, J., Laibson, D., & Madrian, B. (2010). Simplification and saving. Cambridge, MA: National Bureau of Economic Research.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s key objective was to examine the impact of Quick Enrollment, an option enabling individuals to more quickly and easily enroll in a retirement savings plan, on plan enrollment rates and contribution levels at two firms in the United States (companies A and B). The study also examined the impact of Easy Escalation, an option enabling individuals to quickly increase their retirement-plan contribution levels to a predetermined percentage of pay, on contribution rates. The analysis was based on data on enrollment in and contributions to retirement plans from Hewitt Associates, a benefits administration and consulting firm. The study used an interrupted time series (ITS) design, analyzing changes in outcomes before and after the implementation of Quick Enrollment or Easy Escalation. The study found that Quick Enrollment increased enrollment rates by 10 to 20 percentage points and that Easy Escalation increased the percentage of employees contributing to their savings plans at the higher, predetermined rate specified by Easy Escalation.  The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to Quick Enrollment or Easy Escalation. Other factors are likely to have contributed to the changes in enrollment and contribution rates.","the Quick Enrollment and Easy Escalation","Employer provided retirement benefits",Employed,"Health care and social assistance, Manufacturing","United States",2010,,"Behavioral Finance: Retirement Review Protocol" "Retiree health insurance and disengagement from a career job.","Robinson, C., & Clark, R. (2010). Retiree health insurance and disengagement from a career job. Journal of Labor Research, 31(3), 247-262.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-No impacts Employment","Summary: The study examined the effect of availability of retiree health insurance (RHI) on a person’s decision to leave a career job (a measure of retirement). Using data from the Health and Retirement Study (HRS), the authors used a nonexperimental approach to estimate the likelihood of retirement for those who worked at firms with RHI compared with those who worked at firms without RHI. The study found that workers at firms that offer RHI are more likely to retire than those at firms that do not offer RHI. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to the availability at RHI, but other factors might also have contributed.","Retiree Health Insurance","Other health and safety Older workers' programs","Adult, Older worker",,"United States",2010,https://link.springer.com/article/10.1007/s12122-010-9091-4,"Employment and Training Review Protocol" "Implementation and early training outcomes of the High Growth Job Training Initiative: Final report.","Eyster, L., Nightingale, D.S., Barnow, B., O'Brien, C., Trutko, J. & Kuehn, D. (2010) Implementation and early training outcomes of the High Growth Job Training Initiative: Final report. Washington, D.C.: Urban Institute Center on Labor, Human Services, and Population.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the High Growth Job Training Initiative (HGJTI) grant on earnings. This summary focuses on the apprenticeship program at South Texas College (STC). This nonexperimental study used propensity score matching and difference-in-differences models to compare the earnings outcomes of workers who participated in the apprenticeship program to those who did not. The study found that participation in the apprenticeship program was significantly related to higher earnings. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the South Texas College (STC) apprenticeship program; other factors are likely to have contributed.","South Texas College (STC) Advanced Manufacturing Apprenticeship Program","Employment and Training Services Employer Services Employer partnerships Training and Education Capacity building programs Community college education and other classroom training Registered apprenticeship Work based and other occupational training","Adult, Dislocated or displaced worker",,"United States",2010,https://www.urban.org/research/publication/implementation-and-early-training-ou…,"Community College Review Protocol" "Framing Social Security Reform: Behavioral responses to changes in the full retirement age","Behaghel, L., & Blau, D. M. (2010) Framing Social Security Reform: Behavioral responses to changes in the full retirement age. (IZA Discussion Paper No. 5310). Bonn, Germany: IZA.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study examined the impact of changes to the Full Retirement Age (FRA) on labor force exit, Old Age and Survivor’s Insurance (OASI) claims, and retirement. The study uses a statistical model and the data from the Health and Retirement Study (HRS) to compare outcomes of people in birth cohorts who were affected by the change in FRA with those of a comparison group of people who were not. The study found that the change in the FRA was associated with significant increases in the likelihood of claiming OASI and exiting the labor force at the new, later full retirement age. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the change in FRA, but other factors might also have contributed.","changes to the Full Retirement Age (FRA)","Older workers' programs Other employment and reemployment","Older worker",,"United States",2010,https://www.aeaweb.org/articles?id=10.1257/pol.4.4.41,"Employment and Training Review Protocol" "Choice proliferation, simplicity seeking, and asset allocation","Iyengar, S., & Kamenica, E. (2010). Choice proliferation, simplicity seeking, and asset allocation. Journal of Public Economics, 94, 530-539.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Attitudes-Mod/high-Favorable impacts Attitudes","Summary: Laboratory experiments This study used two laboratory experiments to examine the relationship between the number of choices offered and decision making.  Individuals were asked to choose between risky gambles and sure payoffs, with the number of choices randomly determined. The authors found that when individuals were presented with a larger number of choices, they tended to choose simpler payoff options. The quality of the causal evidence presented in this portion of the study is high. This means we are confident that an increase in the number of choices an individual is given causes him or her to choose a less complex payoff option. Analysis of 401(k) data  The study further aimed to determine whether the number of fund options offered by a 401(k) plan influenced the share of assets allocated to a given class of funds (for example, equities or bonds) by plan members. The authors analyzed differences in portfolio allocations based on the number of funds available using regression analysis, controlling for other 401(k), firm, and employee characteristics. The analysis demonstrated that as the number of investment options increased, people tended to allocate more of their 401(k) savings to money market and bond funds at the expense of equity funds. The quality of causal evidence presented in this portion of the study is low. This means we cannot be confident that an increase in the number of funds offered in a retirement plan causes an increase in the share of funds allocated to money market and bond funds and a concurrent decrease in the share of funds allocated to equities.","Changing the Number of Risky Choices","Employer provided retirement benefits",Employed,,"United States",2010,,"Behavioral Finance: Retirement Review Protocol" "Reentry experiences of confined juvenile offenders: Characteristics, service receipt and outcomes of juvenile male participants in the SVORI multi-site evaluation. The multi-site evaluation of the Serious and Violent Offender Reentry Initiative.","Hawkins, S., Lattimore, P., Dawes, D., & Visher, C. (2010). Reentry experiences of confined juvenile offenders: Characteristics, service receipt and outcomes of juvenile male participants in the SVORI multi-site evaluation. The multi-site evaluation of the Serious and Violent Offender Reentry Initiative. NCJ 230423. Research Triangle Park, NC: RTI.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Education and skills gains-Mod/high-Mixed impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment","Summary: The study examined the impact of the Serious and Violent Offender Reentry Initiative (SVORI) on employment, education, and recidivism among juvenile incarcerated males. The study used propensity scores to match participants in SVORI to nonparticipants who were similar on observed characteristics. The study used data from interviews collected at multiple time points through 15 months after release. The study found that SVORI participants were more likely than nonparticipants to be enrolled in school 3 months post-release (68 and 52 percent, respectively) and were more likely than nonparticipants to have a job with benefits 15 months post-release (69 and 40 percent, respectively). The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the SVORI program, but other factors might also have contributed.","Serious and Violent Offender Reentry Initiative (SVORI)","Other employment and reemployment Other health and safety Reentry Substance abuse recovery Youth programs","Youth, Justice-involved, Disconnected youth, Male",,"United States",2010,https://www.ncjrs.gov/pdffiles1/nij/grants/230423.pdf,"Opportunities for Youth Review Protocol" "Can Social Security explain trends in labor force participation of older men in the United States?","Blau, D. M., & Goodstein, R. M. (2010). Can Social Security explain trends in labor force participation of older men in the United States? Journal of Human Resources, 45(2), 328-363.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study examined the impact of changes in Social Security Administration (SSA) retirement rules on men’s labor force participation rates in the 1960s–1980s and 1990s–2000s. The study used regression analyses to predicted labor force participation rates using actual and counterfactual SSA retirement rules. These analyses use data from the Current Population Survey (CPS) and the Survey of Income and Program Participation (SIPP) for the period from 1962 to 2005 merged with data from SSA on taxable earnings. The study found that changes in the delayed retirement credit and full retirement age in 1983 could explain one-quarter to one-half of the increase in labor force participation in the early 2000s. The quality of causal evidence presented in this report is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to SSA reforms, but other factors might also have contributed.","Social Security Administration Reforms","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2010,http://jhr.uwpress.org/content/45/2/328.abstract,"Employment and Training Review Protocol" "Elderly immigrants’ labor supply response to supplemental security income.","Kaushal, N. (2010). Elderly immigrants’ labor supply response to supplemental security income. Journal of Policy Analysis and Management, 29(1), 137-162.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study objective is to examine the impact of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 on employment outcomes of foreign-born people and noncitizens ages 60 to 74 The study used a statistical model and the data from the Current Population Survey’s monthly outgoing rotation group and the March Current Population Survey from 1994 to 2005 to estimate the impact. The study found that the PRWORA is associated with increased employment and decreased retirement for foreign-born men compared with U.S.-born men, but it found no statistically significant association with the employment for foreign-born women. The quality of the causal evidence presented in this report is low because the study is a nonexperimental analysis that does not demonstrate that the groups were similar before the policy change. This means we are not confident that the estimated effects are attributable to PRWORA; other factors are likely to have contributed.","Personal Responsibility and Work Opportunity Reconciliation Act of 1996","Federal retirement benefits Foreign and temporary workers","Adult, Older worker, Immigrant",,"United States",2010,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4222576/,"Employment and Training Review Protocol" "The role of inspection sequence in compliance with the US Occupational Safety and Health Administration’s (OSHA) standards: Interpretations and implications","Ko, K., Mendeloff, J., & Gray, W. (2010). The role of inspection sequence in compliance with the US Occupational Safety and Health Administration's (OSHA) standards: Interpretations and implications. Regulation and Governance, 4(1), 48-70.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Compliance-Low-Mixed impacts Compliance","Summary: The study’s objective was to examine the effect of the sequence and timing of OSHA inspections on compliance with OSHA standards, as measured by the number of violations cited during an inspection. The study used a regression model to compare the number of OSHA violations cited during inspections based on their order during the sample period—for example, during the second compared with the first inspection during the sample period—and for which different lengths of time had passed since the previous inspection. Examining firms inspected multiple times, the study reported that the total number of violations cited in the second inspection was lower than the number of violations cited in the first. In these firms, the number of violations cited was larger for every additional year elapsed since the previous inspection. The quality of causal evidence presented in this study is low. Although it provides interesting descriptive information, the study does not provide evidence of a causal effect of inspections, or their timing, on violations.","OSHA Enforcement Activities",Inspections,Employed,,"United States",2010,,"OSHA Enforcement Review Protocol" "Implementation and early training outcomes of the High Growth Job Training Initiative: Final report","Eyster, L., Nightingale, D.S., Barnow, B., O'Brien, C., Trutko, J. & Kuehn, D. (2010) Implementation and early training outcomes of the High Growth Job Training Initiative: Final report. Washington, D.C.: Urban Institute Center on Labor, Human Services, and Population","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the High Growth Job Training Initiative (HGJTI) grant on earnings. This summary focuses on the Certified Nursing Assistant (CNA) program at Columbia Gorge Community College (CGCC) in Oregon. This nonexperimental study used propensity score matching and difference-in-differences models to compare the earnings outcomes of women who participated in the CNA program to those who did not. The study found that women who participated in the CGCC CNA program earned significantly less than women in the comparison group. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the CGCC CNA program, but other factors might also have contributed.","Columbia Gorge Community College’s (CGCC) Certified Nursing Assistant (CNA) program","Employment and Training Services Employer Services Employer partnerships Training and Education Capacity building programs Community college education and other classroom training Work based and other occupational training","Adult, Dislocated or displaced worker",,"Rural, United States",2010,https://www.urban.org/research/publication/implementation-and-early-training-ou…,"Community College Review Protocol" "The impact of training on the frequency of internal promotion of employees and managers","West, L. S. (2010). The impact of training on the frequency of internal promotion of employees and managers. (Doctoral dissertation). University of North Texas Digital Library. https://digital.library.unt.edu/ark:/67531/metadc30526/m2/1/high_res_d/dissertation.pdf [Manager sample]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the relationship between formal training and promotion for managers. The author investigated similar research questions in another study, the profile of which is available [here]. The other study examined the relationship between formal training and promotion for employees who were not managers. The author used statistical methods in a nonexperimental analysis to estimate impacts of hours of formal training, drawing on archival survey data from the 1996–1997 the National Organizations Survey. The study found no statistically significant relationships between formal training and managers’ promotions. The quality of causal evidence presented in this report is low because the author did not account for existing differences of managers in organizations that offered various hours of formal training. This means we would not be confident that the estimated effects would be attributable to formal training; other factors would have been likely to have contributed. The study did not, however, find statistically significant relationships.","the formal training","Other training and education",Employed,,"United States",2010,https://digital.library.unt.edu/ark:/67531/metadc30526/m2/1/high_res_d/disserta…,"Employment and Training Review Protocol" "Reinvesting in America’s youth: Lessons from the 2009 Recovery Act Summer Youth Employment Initiative.","Bellotti, J., Rosenberg, L., Sattar, S., Esposito, A., & Ziegler, J. (2010). Reinvesting in America’s youth: Lessons from the 2009 Recovery Act Summer Youth Employment Initiative. Princeton, NJ: Mathematica Policy Research.","Opportunities for Youth","Study Type: Implementation Analysis",,,"Summary: The research examines the implementation of the Summer Youth Employment Initiative (SYEI). Funded by the American Recovery and Reinvestment Act of 2009, SYEI aimed to reverse the steep decline in youth employment during the recession by providing employment opportunities for youth in the summer of 2009. Through the initiative, youth were placed in summer work experiences with local employers and their wages were paid with Recovery Act funds. The work experiences were fairly flexible, but had to be age appropriate and lead to youth meeting work readiness goals. The authors analyzed state performance data through December 31, 2009, and data from in-depth site visits to 20 selected sites during July and August 2009. The authors reported that it took a large effort to get the program up and running in such a short amount of time (about four months). Some sites reported having to make compromises along the way, given the quick time frame, but all reported satisfaction with their accomplishments. The initiative was implemented successfully in all the studied sites, with all sites able to recruit a sufficient number of participants and place them in employment. Youth participants and the employers interviewed were very positive about the initative. Although the implementation of SYEI was considered a success, it was not without some initial challenges, including handling the large volume of applicants and determining their eligibility, reaching older youth ages 22 to 24 and veterans and their spouses, and appropriately matching youth to employers.","Summer Youth Employment Initiative (SYEI)","Basic skills Job search assistance and supportive services Work based and other occupational training Youth programs","Youth, Disconnected youth, Low-skilled, Low income",,"United States",2010,http://files.eric.ed.gov/fulltext/ED510398.pdf,"Opportunities for Youth Review Protocol" "St. Mary’s County juvenile drug court outcome and cost evaluation","Mackin, J., Lucas, L., Lambarth, C., Waller, M., Herrera, A., Carey, S., & Finigan, M. (2010). St. Mary’s County juvenile drug court outcome and cost evaluation. Portland, OR: NPC Research.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of participation in a juvenile drug court program in St. Mary’s County, Oregon, on recidivism. The study used administrative data from the departments of Juvenile Services, Public Safety and Correctional Services, and Health and Mental Hygiene to compare outcomes of youth who participated in juvenile drug court with outcomes from a group of similar, drug court-eligible youth who did not participate. The study found no statistically significant effects on recidivism, including total number of rearrests, rearrests for drug charges, and rearrest rates over a two-year period. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we would be somewhat confident that any estimated effects would be attributable to participation in the juvenile drug court, but other factors might also have contributed. However, the study did not find any statistically significant effects.","St. Mary's County Juvenile Drug Court","Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2010,http://www.courts.state.md.us/opsc/dtc/pdfs/evaluationsreports/stmaryscountyjuv…,"Opportunities for Youth Review Protocol" "The influences on bridge employment decisions.","Pengcharoen, C., & Shultz, K. S. (2010). The influences on bridge employment decisions. International Journal of Manpower, 31(3), 322-336. doi:10.1108/01437721011050602","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Unfavorable impacts Employment","Summary: The study examined the impact of work schedule flexibility on retirement outcomes among older workers, where work schedule flexibility was measured as the ability to reduce hours from one’s regular work schedule. The study used a statistical model (multinomial logit regression) and the longitudinal data from the Health and Retirement Study (HRS) from 1992 and 2002 to examine the relationship between reporting work schedule flexibility or inflexibility during the baseline survey and reporting complete retirement, partial retirement, or continued employment at the follow-up survey 10 years later among older workers. The study found that individuals who reported work schedule inflexibility in the 1992 were more likely to report completely retired than partially retired in 2002, but when comparing the likelihood of reporting continued employment to complete or partial retirement, there was no difference between older workers reported work schedule flexibility and those who reported inflexibility in 1992. The quality of causal evidence presented in this study is low because the authors did not demonstrate that the participants reporting work schedule flexibility were similar to the participants reporting work schedule inflexibility in 1992. This means we are not confident that the estimated effects are attributable to work schedule flexibility; other factors are likely to have contributed.",,"Other wages and benefits","Adult, Older worker",,"United States",2010,https://www.researchgate.net/publication/46545640_The_influences_on_bridge_empl…,"Employment and Training Review Protocol" "What kinds of injuries do OSHA inspections prevent? ","Haviland, A., Burns, R., Gray, W., Ruder, T., & Mendeloff, J. (2010). What kinds of injuries do OSHA inspections prevent? Journal of Safety Research, 41(4), 339-345.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine whether OSHA inspections that resulted in penalties or citations prevented specific types of workplace injuries. The study used a regression model to compare the changes in the workplace injuries in single-facility manufacturing firms that had received an inspection that resulted in a penalty or citation to changes in firms that had not received a penalty or citation. The group of firms that had not received a penalty or citation combined two types of firms: firms that had not been inspected and firms that received inspections that did not result in a penalty or citation. The study found that inspections that imposed penalties were associated with statistically significant reductions in the rates of all types of injuries examined. Among the specific citations studied, only citations for violating the general standard for personal protective equipment were associated with reduced injury rates. The quality of the causal evidence presented in this study is low. This means we are not confident that OSHA inspections that resulted in penalties or citations caused the reductions in workplace injuries.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,Manufacturing,"United States",2010,,"OSHA Enforcement Review Protocol" "Effect of job skills training on employment and job seeking behaviors in an American Indian substance abuse treatment sample.","Foley, K., Pallas, D., Forcehimes, A., Houck, J., Bogenschutz, M., Keyser-Marcus, L., & Svikis, D. (2010). Effect of job skills training on employment and job seeking behaviors in an American Indian substance abuse treatment sample. Journal of Vocational Rehabilitation, 33(2010), 181-192.","Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Education and skills gains-Mod/high-Unfavorable impacts Education and skills gains Employment-Mod/high-No impacts Employment","Summary: The study’s objective was to examine the impact of workshops for job seekers on employment and training outcomes. The study used a randomized controlled trial (RCT) design, in which eligible participants were randomly assigned to the treatment group, which could participate in the job seeker workshops, or the control group, which could not participate. The authors used follow-up surveys three and six months after random assignment to measure self-reported employment and training outcomes. The study found that three months after workshop participation, the treatment group spent fewer hours on training activities than the control group; the study did not find any significant impacts of the job seekers’ workshop on the number of hours spent on work or percentage of each group who were employed at the three-month follow-up. The quality of causal evidence presented in this report is high for the percentage employed and hours spent on work and training outcomes collected at the three-month follow-up because those outcomes were based on a well-implemented RCT with low attrition. This means we are confident that the estimated effects are attributable to the job seekers’ workshop, and not to other factors. However, the quality of causal evidence presented in this report is low for the days to employment outcomes at the three-month follow-up and all outcomes at the six-month follow-up. This means we are not confident that the estimated effects are attributable to the job seekers’ workshop; other factors are likely to have contributed.","Job Seekers’ Workshop","Job search assistance and supportive services","Other barriers, Unemployed, American Indian or Alaska Native",,"United States",2010,,"Employment and Training Review Protocol" "The impact of training on the frequency of internal promotion of employees and managers","West, L. S. (2010). The impact of training on the frequency of internal promotion of employees and managers. (Doctoral dissertation). University of North Texas Digital Library. https://digital.library.unt.edu/ark:/67531/metadc30526/m2/1/high_res_d/dissertation.pdf [Employee sample]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the relationship between formal training and promotion for employees who were not managers. The author investigated similar research questions in another study, the profile of which is available [here]. The other study examined the relationship between formal training and promotion for managers. The author used statistical methods in a nonexperimental analysis to estimate impacts of hours of formal training, drawing on archival survey data from the 1996–1997 National Organizations Survey. The study found a statistically significant and positive relationship between typical hours of formal training and the frequency of promotions for employees. The quality of causal evidence presented in this report is low because the author did not account for existing differences of employees in organizations that offered various hours of formal training. This means we are not confident that the estimated effects are attributable to formal training; other factors are likely to have contributed.","the formal training","Other training and education",Employed,,"United States",2010,https://digital.library.unt.edu/ark:/67531/metadc30526/m2/1/high_res_d/disserta…,"Employment and Training Review Protocol" "The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models [TAAG—Medford]","Hendra, R., Dillman, K-N., Hamilton, G., Lundquist, E., Martinson, K., Wavelet, M., Hill, A., & Williams, S. (2010). The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. [TAAG—Medford]","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to estimate the impact of post-employment job supports on former and current public assistance recipients’ employment and benefits receipt outcomes after three years. The Medford site was one of four in Oregon that participated in the Employment Retention and Advancement (ERA) project. The authors randomly assigned 1,164 employed single parents who were former or current public assistance recipients to either a treatment group that received services from Medford’s ERA project (Transition, Advancement, and Growth, or TAAG), or a control group that could participate in other programs in the community. The authors analyzed data from Unemployment Insurance (UI), Temporary Assistance for Needy Families (TANF), and Food Stamps administrative records and a survey administered 12 months after random assignment. The study found that those assigned to the Medford ERA project earned, on average, $41 more per week when surveyed one year after random assignment than those in the control group. However, treatment group members were also 6.7 percentage points more likely to have ever received Food Stamps at the three-year follow-up than those in the control group. The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Medford ERA project and not to other factors.","the Employment Retention and Advancement Project, Medford","Job search assistance and supportive services Other employment and reemployment Unemployment Insurance","Low income, Parent, Employed",,"United States",2010,http://files.eric.ed.gov/fulltext/ED514699.pdf,"Employment and Training Review Protocol" "The effects of changes in state SSI supplements on preretirement labor supply","Neumark, D., & Powers, E. T. (2005). The effects of changes in state SSI supplements on preretirement labor supply. Public Finance Review, 33(1), 3-35.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of a theoretical increase in Supplemental Security Income (SSI) benefits on the preretirement employment outcomes for likely SSI participants ages 60 to 64. Using data from the annual Current Population Survey from 1979 to 2001 and state-level SSI receipt data, the authors used regression methods in a nonexperimental analysis of likely SSI participants to compare the outcomes of workers ages 62 to 64 with those of workers ages 60 to 61. The study found that an increase in monthly SSI benefits was associated with a statistically significant decrease in the employment rate and number of hours worked for likely SSI recipients ages 62 to 64 compared with those ages 60 to 61. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to changes in SSI benefits; other factors are likely to have contributed.","Supplemental Security Income (SSI) benefits","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2010,https://journals.sagepub.com/doi/pdf/10.1177/1091142104270655,"Employment and Training Review Protocol" "The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models [PROGRESS—Eugene]","Hendra, R., Dillman, K-N., Hamilton, G., Lundquist, E., Martinson, K., Wavelet, M., Hill, A., & Williams, S. (2010). The Employment Retention and Advancement project: How effective are different approaches aiming to increase employment retention and advancement? Final impacts for twelve models. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families. [PROGRESS—Eugene]","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to estimate the impact of post-employment job supports on former welfare recipients’ employment and benefits receipt outcomes after three years. The Eugene site was one of four in Oregon that participated in the Employment Retention and Advancement (ERA) project. The authors randomly assigned 1,179 employed single parents who were former Temporary Assistance for Needy Families (TANF) program participants to either a treatment group that received employment retention services from Eugene’s Progress Towards Retention, Opportunities, Growth, Enhancement and Self-Sufficiency (PROGRESS) program or a control group that could participate in other employment-related programs in the community. The authors analyzed data from Unemployment Insurance (UI), Temporary Assistance for Needy Families (TANF), and Food Stamps administrative records and a survey administered 12 months after random assignment. The study found that those in the PROGRESS group were 7 percentage points less likely than control group members to be employed in the third year after random assignment and 6.6 percentage points less likely to have been employed in every quarter of that year. The quality of causal evidence provided in this study is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the estimated effects are attributable to the Eugene ERA program and not to other factors.","the Employment Retention and Advancement Project, Eugene","Other employment and reemployment Unemployment Insurance","Employed, Low income, Parent",,"United States",2010,http://files.eric.ed.gov/fulltext/ED514699.pdf,"Employment and Training Review Protocol" "Anne Arundel County juvenile treatment court outcome and cost evaluation","Mackin, J., Lucas, L., Waller, M., Carey, S., & Finigan, M. (2010). Anne Arundel County juvenile treatment court outcome and cost evaluation. Portland OR: NPC Research.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of a juvenile treatment court in Anne Arundel County, Maryland, on recidivism. The authors used a nonexperimental design, matching youth who participated in the juvenile treatment court with similar court-involved youth who did not participate in the juvenile treatment court. The authors used administrative data to compare the mean number of juvenile rearrests for the study groups over a two-year follow-up period. The study found that 53 percent of the treatment court group, compared with 73 percent of the comparison group, had a juvenile rearrest within 24 months. The study also found differences of a similar magnitude at three intermediate follow-ups (6-, 12-, and 18-month follow-ups). The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the juvenile treatment court, but other factors might also have contributed.","Anne Arundel Juvenile Treatment Court","Other training and education Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2010,http://www.courts.state.md.us/opsc/dtc/pdfs/evaluationsreports/annearundelcount…,"Opportunities for Youth Review Protocol" "Toward reduced poverty across generations: Early findings from New York City’s conditional cash transfer program","Riccio, J., Dechausay, N., Greenberg, D., Miller, C., Rucks, Z., and Verma, N. (2010). Toward reduced poverty across generations: Early findings from New York City’s conditional cash transfer program. New York: MDRC.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Education and skills gains-Mod/high-Favorable impacts Education and skills gains Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of Family Rewards, an experimental privately funded conditional cash transfer program in New York City, on earnings, employment, public benefits receipt, and education. The study was based on a randomized controlled trial and estimated the effect of the Family Rewards program on low-income families. The authors used New York City and New York State administrative data to compare average outcomes among those offered access to the program against the average outcomes of those excluded, after adjusting for chance initial differences between the groups. The study found that, on average, the Family Rewards program decreased the likelihood of ever being employed at a job covered by unemployment insurance (UI) in the first year. However, the Family Rewards program increased the likelihood of being employed at a job not covered by UI after 18 months, total monthly income after 18 months, and the likelihood that adult recipients had received an associate’s degree after 18 months. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Family Rewards program, and not to other factors.","the Family Rewards Program","Other employment and reemployment Other training and education Unemployment Insurance","Unemployed, Low income, Parent",,"United States",2010,http://www.mdrc.org/sites/default/files/FamRewards2010ONYC%20FULL%20Report%20RE…,"Employment and Training Review Protocol" "Return on investment analysis of a selected set of workforce system programs in Indiana. [WIA Dislocated Worker]","Hollenbeck, K. (2009). Return on investment analysis of a selected set of workforce system programs in Indiana. Indianapolis, IN: Indiana Chamber of Commerce Foundation. [WIA Dislocated Worker]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Dislocated Worker program on the employment rate, earnings, and benefit receipt of low-income adults who are dislocated workers. The author used a nonexperimental method to compare the short-term (three quarters after program exit) and long-term (seven quarters after program exit) employment, earnings, and Unemployment Insurance benefits between those who took part in the WIA Dislocated Worker Program relative to those who participated in the WorkOne program. The study found that, compared with those who participated in the WorkOne program, WIA Dislocated Worker Program participants had higher employment and earnings and lower benefit receipt. The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the WIA Dislocated Worker program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Workforce Investment Act (WIA) Dislocated Worker Program","Job search assistance and supportive services Other employment and reemployment Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker, Low income",,"United States",2009,http://www.indianachamber.com/images/media/studies/ROIanalysisWorkforceDevelopm…,"Review Protocol" "Assessment of the impact of WorkSource job search services","Lee, K., Weeks, G., Bodeutsch, G., Clay-Poole, S., Garoflo, T., Petritz, M., Stromsdorfer, E., Field, J., Paterson, T., & Jennings, K. (2009). Assessment of the impact of WorkSource job search services. Washington: Washington State Employment Security Department Labor Market and Economic Analysis.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to assess the effectiveness of Washington State WorkSource job search services. WorkSource services included staff-assisted job matching, provision of labor market information, job search and placement assistance, and initial assessments. Using a nonexperimental design, the authors compared the employment and earnings of Unemployment Insurance (UI) claimants who received at least one WorkSource service with those of UI claimants who did not receive any WorkSource services using state UI administrative records for more than 64,000 Washington State UI claimants with no UI-reported wage earnings in the third quarter of 2007. The study found that WorkSource services were associated with a higher probability of employment and higher earnings growth for men and women who were intermittently (rather than continuously) employed in the baseline period. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to WorkSource services. Other factors are likely to have contributed.","JSA services","RESEA Job search assistance and supportive services","Unemployed, Female, Male",,"United States",2009,https://esdorchardstorage.blob.core.windows.net/esdwa/Default/ESDWAGOV/labor-ma…,"Reemployment Review Protocol" "Optimal defaults and active decisions","Carroll, G., Choi, J., Laibson, D., Madrian, B., & Metrick, A. (2009). Optimal defaults and active decisions. The Quarterly Journal of Economics, 124(4).","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study examined the impact of requiring employees to actively decide whether to enroll in a 401(k) plan, called active-decision enrollment, on newly hired employees’ 401(k) enrollment decisions and contribution rates at a large U.S.-based company. Active-decision enrollment was compared with standard enrollment, which required employees to opt in to a 401(k) plan. The analysis was based on administrative data on enrollment in and contributions to a retirement plan. The study analyzed changes in outcomes before and after the change from active-decision to standard enrollment. The study found that requiring employees to actively opt in or out of savings plans increased 401(k) participation and contribution rates, compared with standard enrollment. The study also found that employees who participated in the 401(k) plan under the active-decision enrollment policy contributed less, on average, than participants under standard enrollment.  The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the company’s active-decision enrollment policies. Other factors are likely to have contributed.","Active-Decision 401(k) Enrollment","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "New evidence on the labor supply effects of the Social Security earnings test.","Friedberg, L., & Webb, A. (2009). New evidence on the labor supply effects of the Social Security earnings test. Tax Policy and the Economy, 23(1), 1-36.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of changes to the Social Security earnings test in 1996 and 2000 on employment outcomes. The authors used a nonexperimental design and the data from the Current Population Survey (CPS) and the Health and Retirement Study (HRS) to compare outcomes before and after the changes in earning test rules for those between the ages of 62 and 74 who faced different earnings test thresholds at different ages and years. The study found that upcoming anticipated earnings test between ages 62 to 69 was associated with a significantly lower likelihood of having a job last week. A higher present value of upcoming anticipated earnings threshold between ages 62 to 69 was associated with a significantly higher likelihood of having a job last week. The quality of causal evidence presented in this study is low because the study is a nonexperimental analysis that did not demonstrate that the groups being compared were similar and did not account for possible differences in the analysis. This means we are not confident that the estimated effects are attributable to the changes in the earnings test decision. Other factors are likely to have contributed.","Social Security Earnings Test","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2009,,"Employment and Training Review Protocol" "New ways to make people sSave: A social marketing approach","Lusardi, A., Keller, P. A, & Keller, A. M. (2009). New ways to make people sSave: A social marketing approach. National Bureau of Economic Research Working Paper 14715. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to assess the effect of a retirement planning aid on encouraging new hires at a large institution to enroll in a supplemental retirement account (SRA). The authors presented results from both an initial pilot test and a full-scale implementation using a modified version of the planning aid. The authors compared the mean 30- and 60-day enrollment rates of cohorts of new hires using administrative data provided by the institution. The final version of the planning aid increased SRA enrollment considerably relative to the control group, with more than 27 percent of new hires enrolling within 30 days and more than 41 percent enrolling within 60 days, compared with 7 and 29 percent, respectively, in the control group. Results were similar during the pilot intervention phase. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to the planning aid; other factors are likely to have contributed.","a Retirement Planning Tool","Employer provided retirement benefits",Employed,,"United States",2009,http://www.nber.org/papers/w14715,"Behavioral Finance: Retirement Review Protocol" "Youth work and learn at Our Piece of the Pie (OPP): Findings of the formative evaluation of Connecticut's youth development model","Britner, P., Buch, R., Colon, R., Milan, S., Singer, M., & Snow, S. (2009). Youth work and learn at Our Piece of the Pie (OPP): Findings of the formative evaluation of Connecticut's youth development model. Storrs, CT: Center for Health, Intervention and Prevention, University of Connecticut.","Opportunities for Youth","Study Type: Descriptive Analysis",,,"Summary: This report describes the work of Our Piece of the Pie (OPP), an organization that gives young people in Hartford, CT, access to a range of services, including academic assistance, vocational training, and guidance from caring adults trained to provide mentorship and other support. The study aimed to gather information on the characteristics of the youth served by OPP, their participation in program activities, and the progress they made in reaching academic and employment goals. The evaluation team conducted direct observations of intervention activities, interviews with program staff and participants, and surveys of participating youth. It also drew on program data collected by OPP. The study found that most participants spent a considerable amount of time with their mentors each week, with some also taking advantage of career advancement and educational preparation services. Although participating youth were making progress on their goals at the time of the evaluation, particularly in achieving educational milestones, the authors suggested that the program could benefit by strengthening participant-mentor relationships, adapting programming for special populations, and improving the quality of program data.","Our Piece of the Pie (OPP)","Job search assistance and supportive services Youth programs Mentoring Other training and education Work based and other occupational training","Youth, Disconnected youth",,"United States",2009,,"Opportunities for Youth Review Protocol" "The repeal of the retirement earnings test and the labor supply of older men.","Engelhardt, G. V., & Kumar, A. (2009). The repeal of the retirement earnings test and the labor supply of older men. Journal of Pension Economics & Finance, 8(4), 429-450.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of the repeal of the earnings test under the Senior Citizens Freedom to Work Act of 2000 on the labor supply and earnings of older men ages 62 to 72. The authors used a nonexperimental approach and data from the Health and Retirement Study to compare labor force participation, annual hours of work, retirement behavior, and earnings of men who were and were not subject to the repeal of the earnings test. The study found that the policy change was associated with a significant increase in older men’s average annual hours of work. The policy change was also associated with a change in the distribution of real earnings in that a smaller proportion of older men had earnings at or below the earnings test threshold. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000; other factors are likely to have contributed.","Senior Citizens Freedom to Work Act of 2000","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2009,,"Employment and Training Review Protocol" "The effects of goal orientation on job search and reemployment: A field experiment among unemployed job seekers","van Hooft, E., & Noordzij, G. (2009). The effects of goal orientation on job search and reemployment: A field experiment among unemployed job seekers. Journal of Applied Psychology, 94(6), 1581-1590.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the effects of goal orientation—the way individuals frame their objectives in a given situation—on job search and reemployment among unemployed job seekers. The study was a randomized controlled trial (RCT) in which participants were assigned to one of three workshops: a learning goal-oriented (LGO) workshop, a performance goal-oriented (PGO) workshop, and a control workshop. The authors compared the job-search and reemployment outcomes of participants in the LGO or PGO workshops to the outcomes of those in the control workshop. The study found that participants in the LGO workshop were 22 percentage points more likely to be reemployed eight weeks after the workshop than were participants in the control workshop, a statistically significant difference. The quality of causal evidence presented in this report is high for the comparison of PGO versus control participants’ reemployment statuses because these outcomes derived from a well-conducted RCT with low attrition. This means we are confident that the estimated effects are attributable to workshop type and not to other factors. However, the quality of causal evidence presented in this report is low for the comparison of LGO and control participants’ job-search behaviors and reemployment statuses and the comparison of PGO and control participants’ job-search behaviors because these outcomes derived from an RCT with high attrition that did not control adequately for potential differences in the analytic samples. This means we are not confident that the estimated effects are attributable to workshop type; other factors are likely to have contributed.","the Workshops Studied","Behavioral Interventions","Unemployed, Adult",,International,2009,,"Behavioral Insights Review Protocol" "Reducing the complexity costs of 401(k) participation through Quick Enrollment","Laibson, D., Choi, J., & Madrian, B. (2009). Reducing the complexity costs of 401(k) participation through Quick Enrollment. Developments in the Economics of Aging, 57-82.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to examine the effect of a simplified 401(k) enrollment procedure called Quick Enrollment on plan participation through three trials at two anonymous companies (two trials at company A and one at company B). Quick Enrollment offered employees the choice of joining a 401(k) plan at a fixed contribution rate, investing in prespecified funds. Hewitt Associates, a consultancy and benefits management company, provided data on 401(k) plan participation, compensation, and other demographic characteristics for employees at two anonymous firms. For company A, year-end cross-sectional data from 2002 and 2003 were available for all employees in addition to a September 1, 2004, snapshot. For company B, only year-end data for 2002 and 2003 were available. Plan enrollment rates for new employees rose after each Quick Enrollment implementation, with an enrollment increase of 14 percentage points at company A and 2 percentage points at company B. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to Quick Enrollment; other factors are likely to have contributed.","Quick Enrollment","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "Mental accounting in portfolio choice: Evidence from a flypaper effect","Choi, J., Laibson, D., & Madrian, B. (2009). Mental accounting in portfolio choice: Evidence from a flypaper effect. American Economic Review, 99(5), 2085-2095.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Unfavorable impacts Employer benefits receipt","Summary: The study examined the impact of requiring employees at a large, U.S.-based company to actively choose the asset allocation for an employer’s matching 401(k) contributions, rather than automatically allocating such funds to the employer’s stock. The analysis was based on administrative data on employees’ demographic characteristics and 401(k) contributions. The study analyzed changes in outcomes that occurred when a firm began requiring employees to choose the funds to which the employer’s 401(k) matching contributions would be allocated.  The study found that allowing employees to choose the asset allocation of their matched contributions reduced contributions to employer stock by 64 percentage points in the first year of the new policy. The study also found that employees’ own contribution allocations did not change significantly after the policy was enacted, suggesting that employees made their personal allocation decisions without considering the employer’s match allocation. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the active-decision requirement for employer matched contributions. Other factors are likely to have contributed.","Active Portfolio Choice for Employer Matching Contributions","Employer provided retirement benefits",Employed,,"United States",2009,,"Behavioral Finance: Retirement Review Protocol" "Return on investment analysis of a selected set of workforce system programs in Indiana. [TAA]","Hollenbeck, K. (2009). Return on investment analysis of a selected set of workforce system programs in Indiana. Indianapolis, IN: Indiana Chamber of Commerce Foundation. [TAA]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Trade Adjustment Assistance (TAA) Program on the employment, earnings, and benefit receipt of dislocated workers in Indiana. The author used a nonexperimental method to compare the short-term (three quarters after program exit) and long-term (seven quarters after program exit) employment and earnings between those who took part in the TAA Program relative to those who participated in the WorkOne program. The study found that, compared with those who participated in the WorkOne program, TAA Program participants had higher employment and lower benefit receipt. The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the TAA Program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Trade Adjustment Assistance (TAA) Program","Adult and Dislocated Worker programs Job search assistance and supportive services","Dislocated or displaced worker",,"United States",2009,http://www.indianachamber.com/images/media/studies/ROIanalysisWorkforceDevelopm…,"Review Protocol" "An empirical study of the effects of Social Security reforms on benefit claiming behavior and receipt using public-use administrative microdata.","Benítez-Silva, H., & Yin, N. (2009). An empirical study of the effects of Social Security reforms on benefit claiming behavior and receipt using public-use administrative microdata. Social Security Bulletin, 69(3), 77-95.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study examined the impact of three policy changes to the Social Security program that went into effect in 2000 on average monthly benefits and the proportion of new claimants by age. The study was a nonexperimental analysis that examined outcomes for older workers by age and year using data from the 2004 Old Age, Survivors, and Disability Insurance program public-use microdata files. The study found that the changes were associated with lower average monthly benefits for individuals ages 66 to 69 after the year 2000 relative to individuals of the same ages in 1994. The quality of the causal evidence presented in this report is low because the authors did not demonstrate that the groups were similar before the intervention. This means we are not confident that the estimated effects are attributable to the intervention program; other factors are likely to have contributed.","Policy Changes to the Social Security Program","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2009,https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1480653,"Employment and Training Review Protocol" "The return to firm investments in human capital","Almeida, R., & Carneiro, P. (2008). The return to firm investments in human capital. World Bank Social Protection and Labor Discussion Paper No. 0822.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Productivity-Low-Favorable impacts Productivity","Summary: The study’s objective was to examine the returns to firms’ investments in training for their workers. The authors used data from an annual survey conducted by the Portuguese Ministry of Employment, which is a mandatory survey for all Portuguese firms with at least 100 employees, to determine the internal rate of return to the firm of an additional hour of training per employee. The study found that increasing the amount of training per employee by 10 hours per year led to an increase in current productivity of 0.6 percent. The quality of causal evidence presented in this report is low because the instruments used in the estimation approach were weak, as described by the authors. This means we are not confident that the estimated effects are attributable to the training provided to the employees; other factors are likely to have contributed.",,"Work based and other occupational training",Employed,,International,2008,http://siteresources.worldbank.org/SOCIALPROTECTION/Resources/SP-Discussion-pap…,"Employment and Training Review Protocol" "Workforce Investment Act non-experimental net impact evaluation final report","Heinrich, C., Mueser, P., Troske, K., & Benus, J. (2008). Workforce Investment Act non-experimental net impact evaluation final report. Columbia, MD: IMPAQ International, LLC.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages","Summary: The report’s objective was to examine the effects of participation in the Workforce Investment Act (WIA) Adult and Dislocated Worker programs on average earnings and employment. Results for earnings of participants ages 22 to 26 were presented separately and are the focus of this review. The effectiveness of the programs was evaluated using a matched comparison group design. Outcomes were examined up to 10 quarters after WIA participants entered the Adult or Dislocated Worker programs. The study found that participants in the WIA Adult Program under age 26 earned significantly more than those in the comparison group every quarter after program entry. In the WIA Dislocated Worker program, participants under age 26 began earning significantly more than those in the comparison group a year following program entry. The quality of the causal evidence presented in this study is moderate. This means we have confidence that the effects estimated in this study are attributable at least in part to WIA programs. However, as in any nonexperimental study, other factors not accounted for in the analysis might have contributed to the estimated effects.","the Workforce Investment Act (WIA)","Basic skills Community college education and other classroom training Job search assistance and supportive services Other training and education Work based and other occupational training","Adult, Youth, Dislocated or displaced worker, Female, Male",,"United States",2008,http://www.nawdp.org/Content/NavigationMenu/ResearchReports/2009-10-WIANon-Expe…,"Opportunities for Youth Review Protocol" "Reemployment and Eligibility Assessment (REA) Study: FY 2005 Initiative: final report","Benus, J., Poe-Yamagata, E., Wang, Y., & Blass, E. (2008). Reemployment and Eligibility Assessment (REA) Study: FY 2005 Initiative: final report. Columbia, MD: IMPAQ International. [North Dakota sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) Initiative in North Dakota, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on UI benefits receipt, wages, and employment. The study randomly assigned nearly 2,000 eligible claimants to either the treatment group, which could receive REA services, or the control group, which could not access the same REA services. Those in the treatment group were required to participate in the services to retain their UI eligibility. The authors estimated the program’s effectiveness by analyzing state administrative records on UI payments and information from a follow-up survey of a subsample of study participants. The study found no statistically significant impacts of REA on UI benefits receipt, employment, or earnings. The quality of causal evidence presented in this report is high for the UI benefits receipt outcomes because it is based on a well-implemented randomized controlled trial. For the employment and wages outcomes, the quality of evidence presented in this report is moderate.","JSA services","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2008,http://wdr.doleta.gov/research/FullText_Documents/Reemployment%20and%20Eligibil…,"Reemployment Review Protocol" "An experimental evaluation of teen courts","Stickle, W., Connell, N., Wilson, D., & Gottfredson, D. (2008). An experimental evaluation of teen courts. Journal of Experimental Criminology, 4(2), 137-163.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of participation in a teen court on youth offenders’ rates of recidivism. The study was a randomized controlled trial in which eligible youth were randomly assigned to either the treatment group, which participated in a teen court, or to the control group, which participated in the traditional juvenile justice system. The authors compared outcomes for the two groups using administrative data. The study reported no differences in recidivism outcomes for teen court youth and those in the traditional juvenile justice system. The quality of causal evidence presented in this report is moderate for some outcomes and low for other outcomes. This means we have little confidence that any estimated effects would be attributable to the teen court program. However, the study did not find statistically significant effects.","Teen Court","Youth programs","Youth, Justice-involved, Disconnected youth",,"United States",2008,,"Opportunities for Youth Review Protocol" "The impact of reentry services on juvenile offenders’ recidivism","Bouffard, J., & Bergseth, K. (2008). The impact of reentry services on juvenile offenders’ recidivism. Youth Violence and Juvenile Justice, 6(3), 295-318.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence",,"Summary: The study examined the impact of structured after-care and reentry services for justice-involved youth on their recidivism rates. The study used a nonexperimental design to compare youth who received reentry services to a comparison group of youth who received traditional services in a neighboring county. Data came from the local juvenile court databases, probation and transitional coordinator files, and electronic records used by case managers to track client contacts. The study found that there were no statistically significant differences between study groups in time to first reoffense or in number of criminal contacts with the justice system. The quality of causal evidence presented in this report is low because all treated youth were in one county and all comparison youth were in another. This means we are not confident that the estimated effects are attributable to the after-care and reentry program; other factors are likely to have contributed.","After-care and Reentry Services","Mentoring Reentry Youth programs","Youth, Justice-involved, Disconnected youth",,"United States, Rural",2008,,"Opportunities for Youth Review Protocol" "Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). [WIA-Adult]","Hollenbeck, K., & Huang, W-J. (2008). Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [WIA-Adult]","Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Title I program for adults (including dislocated workers) on participants’ employment and credential completion rates. The study used administrative records to compare the outcomes for low-income adults who took part in the WIA program with a nonexperimental-matched group of adults who did not take part in the program. The study found that the WIA program participants had higher employment and credential completion rates than those of people who did not participate in the program. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the WIA Title I program for adults (including dislocated workers); other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Workforce Investment Act (WIA) Title I Program for Adults","Adult and Dislocated Worker programs Job search assistance and supportive services Other employment and reemployment Other training and education Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Unemployed, Low income",,"United States",2008,http://research.upjohn.org/cgi/viewcontent.cgi?article=1027&context=up_technica…,"Employment and Training Review Protocol" "Can students in technology entrepreneurship courses help foster start-ups by the unemployed?","Watkins, T., Russo, J., & Ochs, J. (2008). Can students in technology entrepreneurship courses help foster start-ups by the unemployed? Journal of Small Business and Enterprise Development, 15(2), 348-364.","Entrepreneurship and Self-Employment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of participating in the Integrated Product Development (IPD) program on dislocated workers’ entrepreneurial activities. The authors used data from a survey administered after one year of program participation to compare the prevalence of entrepreneurial activities among IPD participants compared with program applicants who were not selected to participate in IPD. The study found statistically significant positive relationships between participating in the IPD program and several business development activities and outcomes. The quality of causal evidence presented in this report is low because the authors did not demonstrate that the IPD and comparison groups were comparable before the program, nor did they control for potential differences in their analysis. This means we are not confident that the estimated effects are attributable to the IPD program; other factors are likely to have contributed.","the Integrated Product Development (IPD) Program","Mentoring Other training and education","Self-employed, Dislocated or displaced worker",,"United States",2008,,"Employment and Training Review Protocol" "Workforce Investment Act non-experimental net impact evaluation. [Dislocated worker sample]","Heinrich, C., Mueser, P., & Troske, K. (2008). Workforce Investment Act non-experimental net impact evaluation. Columbia, MD: IMPAQ International. [Dislocated worker sample]","Apprenticeship and Work-Based Training, Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the Workforce Investment Act (WIA) Dislocated Worker Program’s impact on workers’ employment and earnings in 12 states. The authors analyzed administrative data to compare the employment and earnings outcomes of people receiving different tiers of WIA Dislocated Worker Program services with two different matched comparison groups for 16 quarters after program entry. The study found that WIA Dislocated Worker Program services had positive impacts on earnings and employment in most quarters examined. However, those WIA Dislocated Worker Program participants who received core and/or intensive services had significantly better earnings and employment outcomes in the first seven quarters than those who also received training. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Dislocated Worker Program services, but other factors might also have contributed. The report also examined the impacts of WIA Adult Program services; that profile is available here.","the Workforce Investment Act (WIA) Dislocated Worker Program","Other employer services Adult and Dislocated Worker programs Job search assistance and supportive services Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Other training and education","Dislocated or displaced worker, Unemployed",,"United States",2008,https://wdr.doleta.gov/research/FullText_Documents/Workforce%20Investment%20Act…,"Apprenticeship and Work-Based Training, Job Search Assistance Review Protocol" "The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data.","Haider, S. J., & Loughran, D. S. (2008). The effect of the social security earnings test on male labor supply: New evidence from survey and administrative data. Journal of Human Resources, 43(1), 57-87.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Mixed impacts Earnings and wages Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective is to examine the impact of 1983 and 2000 policy changes that removed the Social Security earnings test for certain age groups on the earnings and employment outcomes of affected men The study is a nonexperimental analysis that examines the correlations between the 1983 and 2000 policy changes and earnings and employment status. The study compares the outcomes of men affected by the policy changes with the outcomes of men of similar ages not affected by the policy changes. The study finds that the 1983 policy change is associated with few changes to the observed outcomes and that the 2000 policy change is associated with higher earnings and hours worked per week for affected men. The quality of causal evidence presented in this report is low because the study does not demonstrate that men affected by the policy changes are similar to the men not affected by the policy change, nor does it control for possible differences. This means we are not confident that the estimated effects are attributable to the 1983 and 2000 policy changes; other factors are likely to have contributed.","Social Security Earnings Test","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2008,http://jhr.uwpress.org/content/43/1/57.refs,"Employment and Training Review Protocol" "Reemployment and Eligibility Assessment (REA) Study: FY 2005 Initiative: final report","Benus, J., Poe-Yamagata, E., Wang, Y., & Blass, E. (2008). Reemployment and Eligibility Assessment (REA) Study: FY 2005 Initiative: final report. Columbia, MD: IMPAQ International. [Minnesota sample]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of the Reemployment and Eligibility Assessment (REA) initiative in Minnesota, a program that provided eligibility and reemployment case management services to Unemployment Insurance (UI) claimants, on UI benefits receipt, wages, and employment. The study randomly assigned 5,898 eligible claimants to one of two treatment groups, which received REA services of varying intensity, or the control group. Those in the treatment groups were required to participate in the services to retain their UI eligibility. The authors estimated the program’s effectiveness by analyzing state administrative records on UI payments and information from a follow-up survey of a subsample of study participants. The study found that UI claimants in the more-intensive treatment group claimed fewer weeks of UI benefits and were less likely to have an overpayment detected. There were no significant impacts on employment or earnings for either treatment group. The quality of causal evidence presented in this report is moderate because it is a randomized controlled trial with different selection processes for the treatment and control groups, but the authors controlled sufficiently for baseline characteristics in the analysis. This means we have some confidence that the estimated effects are attributable to REA, although other factors might also have contributed.","the Reemployment and Eligibility Assessment (REA) Initiative in Minnesota","Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",2008,http://wdr.doleta.gov/research/FullText_Documents/Reemployment%20and%20Eligibil…,"Reemployment Review Protocol" "Evaluation of the Suffolk County juvenile treatment court: Process and impact findings","Kralstein, D. (2008). Evaluation of the Suffolk County juvenile treatment court: Process and impact findings. New York: Center for Court Innovation.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence",,"Summary: The study’s objective was to examine the impact of Suffolk County’s juvenile drug court on recidivism. The study used propensity scores based on state and county administrative records to match juveniles who participated in the drug court to similar juveniles in the year before the introduction of the drug court. The author used regression analysis to estimate the relationship between participation in the juvenile drug court and the number and rate of arrests 18 and 24 months following the initial petition. The study found no statistically significant relationship between the juvenile drug court and recidivism. The quality of causal evidence presented in this report is low because the study did not account for other changes that could have taken place at the same time as the introduction of the drug court and affected juvenile recidivism. This means we are not confident that any estimated effects would be attributable to the juvenile drug court, and not other factors. However, the study found no statistically significant effects.","Suffolk County’s Juvenile Drug Court","Reentry Substance abuse recovery Youth programs","Youth, Justice-involved, Disconnected youth",,"United States",2008,http://www.courtinnovation.org/sites/default/files/Suffolk_JTC.pdf,"Opportunities for Youth Review Protocol" "Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). [TAA]","Hollenbeck, K., & Huang, W.-J. (2008). Workforce program performance indicators for the Commonwealth of Virginia. (Upjohn Institute Technical Report No. 08-024). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [TAA]",,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Education and skills gains-Low-Favorable impacts Education and skills gains Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the impact of the Training Adjustment Assistance (TAA) program on employment and credential completion. The study used administrative records to compare outcomes of low-income adults who took part in the TAA program with outcomes of a nonexperimental matched group of adults who did not take part in the program. The study found that TAA participants had a lower employment rate but were more likely to have obtained a training certificate compared with those who did not participate in the program. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to TAA; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Trade Adjustment Assistance (TAA) Program","Job search assistance and supportive services Other training and education","Low-skilled, Low income, Dislocated or displaced worker",,"United States",2008,http://research.upjohn.org/cgi/viewcontent.cgi?article=1027&context=up_technica…,"Review Protocol" "State age protection laws and the Age Discrimination in Employment Act.","Lahey, J. State age protection laws and the Age Discrimination in Employment Act. (2008). The Journal of Law and Economics, 51(3), 433-460.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-Unfavorable impacts Employment","Summary: The study examined the impact of enforcement of age discrimination laws on the employment and earnings of older male workers The author used a nonexperimental regression model and data from the Current Population Survey to estimate the impact. The study found that males older than 50 in states with age discrimination laws were 0.2 percentage points less likely to be hired compared with those in states without their own age discrimination laws. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the state laws on age discrimination, but other factors might also have contributed.","Age Discrimination Employment Act (ADEA)","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker, Male",,"United States",2008,https://www.jstor.org/stable/10.1086/589670?seq=1#page_scan_tab_contents,"Employment and Training Review Protocol" "Workforce Investment Act non-experimental net impact evaluation. [Adult sample]","Heinrich, C., Mueser, P., & Troske, K. (2008). Workforce Investment Act non-experimental net impact evaluation. Columbia, MD: IMPAQ International. [Adult sample]","Apprenticeship and Work-Based Training, Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the Workforce Investment Act (WIA) Adult Program’s impact on workers’ employment and earnings in 12 states. The authors analyzed administrative data to compare the employment and earnings outcomes of people receiving different tiers of WIA Adult Program services with two different matched comparison groups for 16 quarters after program entry. The study found that WIA Adult Program services had positive impacts on earnings and employment in most quarters examined. Those WIA Adult Program participants who received training earned more and were more likely to be employed than those who received only core and intensive services. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the WIA Adult Program services, but other factors might also have contributed. The report also examined the impacts of WIA Dislocated Worker services; that profile is available here.","the Workforce Investment Act (WIA) Adult Program","Adult and Dislocated Worker programs Job search assistance and supportive services Other employment and reemployment Wagner-Peyser Act Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Other training and education","Unemployed, Low income",,"United States",2008,https://wdr.doleta.gov/research/FullText_Documents/Workforce%20Investment%20Act…,"Apprenticeship and Work-Based Training, Job Search Assistance Review Protocol" "Evaluation of the Boys & Girls Clubs of America Targeted Re-Entry Initiative","Barton, W., Jarjoura, G., & Rosay, A. (2008). Evaluation of the Boys & Girls Clubs of America Targeted Re-Entry Initiative. Indianapolis: Indiana University School of Social Work, and Anchorage: Justice Center, University of Alaska Anchorage.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence",,"Summary: The study’s objective was to examine the impact of the Boys & Girls Clubs of America’s Targeted Re-Entry Initiative on youth’s recidivism. The authors used administrative data to match Initiative participants to a comparison group of nonparticipants released from the same rehabilitation facilities. The authors compared recidivism rates among program participants and the comparison group. The authors reported a statistically significant relationship between participation in the Targeted Re-Entry Initiative and higher rates of re-arrest at one of the three sites studied, and no statistically significant correlations at the other two sites. The quality of causal evidence presented in this report is low because the authors did not include sufficient controls for pre-existing differences between the study groups in their analysis. This means we are not confident that the estimated effects are attributable to the Boys & Girls Clubs of America Targeted Re-Entry Initiative; other factors are likely to have contributed.","the Boys & Girls Clubs of America Targeted Re-Entry Initiative","Basic skills Health Job search assistance and supportive services Other training and education Reentry Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2008,http://justice.uaa.alaska.edu/research/2000/0411.targeted_reentry/0411.05.targe…,"Opportunities for Youth Review Protocol" "Supported employment for middle-aged and older people with schizophrenia.","Twamley, E., Narvaez, J., Becker, D., Bartels, S., & Jeste, D. (2008). Supported employment for middle-aged and older people with schizophrenia. American Journal of Psychiatric Rehabilitation, 11(1), 76-89.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study examined the impact of Individual Placement and Support (IPS) on competitive employment for older people with schizophrenia The study was a randomized controlled trial (RCT) and used weekly work logs cross-referenced with pay stubs to measure employment outcomes over a 12-month period. The study found that IPS increased competitive employment and earnings compared with conventional services. The quality of causal evidence presented in this report is low because the RCT has a confounding factor. This means we are not confident that the estimated effects are attributable to the IPS program; other factors are likely to have contributed.","Individual Placement and Support and Vocational Rehabilitation Program","Health Individuals facing barriers to employment Job search assistance and supportive services","Adult, Older worker, Disability",,"United States",2008,https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2638571/,"Employment and Training Review Protocol" "McNeil Education, Training and Evaluation (2008). Process evaluation of the Demand-Side Youth Offender Demonstration Project (Phase II).","McNeil Education, Training and Evaluation (2008). Process evaluation of the Demand-Side Youth Offender Demonstration Project (Phase II). Chapel Hill, NC: McNeil Education.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of being matched to employment through the Demand-Side Youth Offender Demonstration Project (DSYODP) Phase I on youths’ average weekly earnings. The study compared the wages of employed youth who were matched to employment through DSYODP with the wages of employed youth who were eligible for the DSYODP but had secured employment through other means. The study found that average weekly wages of youth matched to employment through DSYODP were significantly higher than wages of youth who secured employment through other means. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the DSYODP; however, other factors might also have contributed.","Demand-Side Youth Offender Demonstration Project (DSYODP)","Job search assistance and supportive services Youth programs","Youth, Justice-involved, Disconnected youth",,"United States",2008,https://wdr.doleta.gov/research/FullText_Documents/Process%20Evaluation%20of%20…,"Opportunities for Youth Review Protocol" "Effects of public policies on the disposition of pre-retirement lump-sum distributions: Rational and behavioral influences","Burman, L., Coe, N., Dworsky, N., & Gale, W. (2008). Effects of public policies on the disposition of pre-retirement lump-sum distributions: Rational and behavioral influences. CentER Discussion Paper Series No. 2008-94.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine whether tax policies that differently frame the rollover of lump-sum distributions (LSDs) from defined-contribution retirement plans into tax-preferred funds affected rollover behavior. The authors investigated two tax reforms, the first in 1986 and the second in 1993. The 1986 reform levied an additional tax of 10 percent on LSDs (framed as a penalty) that were not rolled over into tax-preferred funds. The 1993 reform required LSDs to provide an automatic rollover option and increased tax withholding rates on cash LSDs. The authors used data from the 1993 Current Population Survey Employee Benefit Supplement to analyze the effects of the 1986 tax reform and data from the 1992–2004 Health and Retirement Study to assess the effect of the 1993 tax reform. Treating the tax rate on LSDs as a penalty in the 1986 reform framework increased LSD rollovers by 16 to 32 percentage points for the average individual in the study. The 1993 reforms led to a 10 to 13 percentage point increase in rollovers for the average individual in the study. The quality of causal evidence provided in this study is low. This means that we are not confident that the estimated effects are attributable to the intervention alone; other factors are likely to have contributed.","the 1986 Tax Reform Act and 1993 Addendum","Federal retirement benefits",Employed,,"United States",2008,,"Behavioral Finance: Retirement Review Protocol" "How did the elimination of the US earnings test above the normal retirement age affect labour supply expectations?","Michaud, P., & Van Soest, A. (2008). How did the elimination of the US earnings test above the normal retirement age affect labour supply expectations? Fiscal Studies, 29(2), 197-231. doi:10.1111/j.1475-5890.2008.00073.x","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The authors used a nonexperimental approach and the data from Health and Retirement Study data from 1996 to 2002 and Social Security earnings history to estimate the impact. The study examined the impact of repealing the earnings test from Social Security recipients between full retirement age and age 70 on older workers’ subjective probability of retirement. The authors used a nonexperimental approach and the data from Health and Retirement Study data from 1996 to 2002 and Social Security earnings history to estimate the impact. The study found that after the repeal of the earnings test, the subjective probability of working of full-time past age 65 for older male workers who were likely to have most of their projected Social Security benefits reduced under the earnings test has increased their expected probability of working relative to those who were not likely to be affected. •The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design.This means we are somewhat confident that the estimated effects are attributable to the repeal of the earnings test, but other factors might also have contributed.","Social Security Earnings Test","Federal retirement benefits Older workers' programs","Adult, Older worker, Male",,"United States",2008,,"Employment and Training Review Protocol" "Labor market shocks and retirement: Do government programs matter?","Coile, C., & Levine, P. (2007). Labor market shocks and retirement: Do government programs matter? Journal of Public Economics, 91(10), 1902-1919.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-No impacts Employment Public benefits receipt-Low-No impacts Public benefit receipt","Summary: The study examined the impact of the generosity of unemployment insurance (UI) benefits on retirement decisions. The study used a statistical model and data from the March Current Population Survey for 1980–2004 to estimate impacts. The study found no relationship between the generosity of UI benefits and the likelihood of retirement or receipt of UI. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before changes in the generosity of UI benefits were observed. This means we are not confident that the estimated effects are attributable to changes in the generosity of UI benefits; other factors are likely to have contributed.","Unemployment Insurance (UI)","Unemployment Insurance","Adult, Older worker, Unemployed",,"United States",2007,http://academics.wellesley.edu/Economics/Coile/PDF%20files/coile-levine%20JPubE…,"Employment and Training Review Protocol" "The Employment Retention and Advancement project: Results from Minnesota’s Tier 2 program","LeBlanc, A., Miller, C., Martinson, K., & Azurdia, G. (2007). The Employment Retention and Advancement project: Results from Minnesota’s Tier 2 program. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study’s objective was to determine the 18-month impact of Minnesota’s Tier 2 welfare-to-work program compared with existing Tier 1 services. Minnesota’s Tier 2 program was part of the nationwide Employment Retention and Advancement (ERA) project. The authors randomly assigned approximately 1,700 long-term Temporary Assistance for Needy Families (TANF) recipients to either a treatment group, which received Tier 2 services including intensive assessments and case management, or a control group, which received Minnesota’s existing Tier 1 TANF services. The authors analyzed data from Minnesota Unemployment Insurance (UI), TANF, and Food Stamps administrative records. The study found that, 12 months after random assignment, more people in the Tier 2 group (79.2 percent) reported that they received Food Stamps than in the control group (70.0 percent). After 18 months however, there were no statistically significant differences between treatment and control group members’ employment, earnings, or benefits receipt. The quality of causal evidence provided in this study is high because it is based on a well-conducted randomized controlled trial. This means we are confident that any estimated effects are attributable to Minnesota’s Tier 2 program and not to other factors.","the Employment Retention and Advancement project, Minnesota’s Tier 2 Program","Job search assistance and supportive services Other employment and reemployment","Unemployed, Low income, Parent",,"United States",2007,http://www.acf.hhs.gov/sites/default/files/opre/minnesota_tier2.pdf,"Employment and Training Review Protocol" "Reentry services: An evaluation of a pilot project in Clay County, MN","Bergseth, K., & McDonald, T. (2007). Reentry services: An evaluation of a pilot project in Clay County, MN. Fargo, ND: North Dakota State University, Department of Criminal Justice and Political Science.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence",,"Summary: The study’s objective was to examine the impact of the Reentry Services Project (RSP) on juvenile recidivism. The authors estimated regression models comparing the number of criminal and official contacts with police or the courts, pulled from an electronic Court Services Tracking System, between RSP and comparison group members, controlling for differences in demographics and number of prior charges. The study found that RSP group members had significantly fewer official and criminal contacts than the comparison group one year after release from out-of-home placement. The quality of causal evidence presented in this report is low because the program was implemented in only one county and there was no variation in the implementation of the program over time. This means we are not confident that the estimated effects are attributable to the RSP; other factors are likely to have contributed.","the Reentry Services Project (RSP)","Health Job search assistance and supportive services Reentry Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2007,http://www.claycountycollaborative.org/projects/RSP%20Final%20Report%202007.pdf,"Opportunities for Youth Review Protocol" "The effects of an experimental intensive juvenile probation program on self-reported delinquency and drug use","Lane, J., Turner, S., Fain, T., & Sehgal, A. (2007). The effects of an experimental intensive juvenile probation program on self-reported delinquency and drug use. Journal of Experimental Criminology, 3(3), 201-219.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence",,"Summary: The study’s objective was to examine the impact of the South Oxnard Challenge Project (SOCP), an alternative juvenile probation program, on delinquency and drug use. The study was a randomized controlled trial. Eligible youth were randomly assigned to either a treatment group, which could participate in SOCP, or a control group, which had routine juvenile probation. The study found that significantly more SOCP youth than youth in the control group reported committing a violent crime in the 12 months after starting probation (67.6 versus 55.6 percent). However, SOCP youth indicated they used ecstasy/MDMA less frequently than control youth in the past 30 days (1.8 versus 4.2 days). The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the SOCP, and not to other factors.","South Oxnard Challenge Project (SOCP)","Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2007,,"Opportunities for Youth Review Protocol" "Individual account investment options and portfolio choice: Behavioral lessons from 401(K) plans","Brown, J., Lang, N., & Weisbenner, S. (2007). Individual account investment options and portfolio choice: Behavioral lessons from 401(K) plans. National Bureau of Economic Research working paper No. 13169. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study aimed to determine whether the number of fund options offered within a given asset class (for example, domestic equities) by a 401(k) plan influences the types of investments made by plan members. The authors analyzed differences in portfolio allocations based on the number of funds available in these asset classes using regression analysis, controlling for other 401(k) and firm characteristics. The analysis demonstrated that as the number of investment options within an asset class increase, individuals tend to allocate more of their 401(k) savings to funds within that class. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that an increase in the number of funds offered within a class causes the increase in the share of funds allocated to assets in that class. We can only say that the two variables of interest are positively correlated.","Varying the Number of Risky Choices","Employer provided retirement benefits",Employed,,"United States",2007,http://www.nber.org/papers/w13169,"Behavioral Finance: Retirement Review Protocol" "A tax on work for the elderly: Medicare as a secondary payer.","Goda, G. S., Shoven, J. B., & Slavov, S. N. (2007). A tax on work for the elderly: Medicare as a secondary payer. (NBER Working Paper No. 13383). Cambridge, MA: National Bureau of Economic Research.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of Medicare as a Secondary Payer (MSP) on older workers’ labor force participation and full-time employment The study used a nonexperimental regression design. The authors used data from the 1980 to 2006 March Supplement of the Current Population Survey. The study found that labor force participation for older males increased following the enforcement of the MSP policy, and that older males and females in the labor force were more likely to be employed full time after the policy was enforced. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention, and confounding factors could have impacted older workers differentially during the time periods examined. This means we are not confident that the estimated effects are attributable to MSP; other factors are likely to have contributed.","Medicare as a Secondary Payer","Federal retirement benefits Older workers' programs","Adult, Older worker",,"United States",2007,https://www.nber.org/papers/w13383,"Employment and Training Review Protocol" "Choice architecture and retirement savings plans","Benartzi, S., Peleg, E., & Thaler, R. (2007). Choice architecture and retirement savings plans. Los Angeles, Ca. SSRN working paper.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The report’s objective was to examine the impact of choice architecture—that is, aspects of a savings plan’s structure and enrollment process—on 401(k) savings decisions. The authors discussed five original, observational studies. Two studies conducted regression analyses using survey data. One used an interrupted time series design and data from Vanguard on employees at 13 companies. Two additional studies conducted regression analyses using Vanguard data on 1.5 million participants in 1,830 of Vangaurd’s defined contribution retirement plans. The study receiving a moderate rating (study 5) found that members of retirement plans that offer retirement date funds, relative to members of retirement plans that offer risk-based funds or neither, held comparatively more equity when they were younger and comparatively less equity when they were older. The quality of causal evidence presented in this report is moderate because study 5 was a quasi-experimental design with adequate controls. We must note, however, that the remaining four studies provide causal evidence rated as low. The overall moderate evidence rating means we are somewhat confident that the estimated effects are attributable to the employees’ access to asset allocation funds, but other factors might also have contributed. In the case of the remaining four studies, however, we are not confident that the estimated effects are attributable to the interventions analyzed. Other factors are likely to have contributed.","the Choice Architecture of Retirement Savings Plans","Employer provided retirement benefits",Employed,,"United States",2007,http://164.67.163.139/Documents/areas/fac/accounting/Benartzi_ChoiceArchitectur…,"Behavioral Finance: Retirement Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). [WIA Adult]","Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [WIA Adult]","Apprenticeship and Work-Based Training, Job Search Assistance","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Unfavorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Adult Program on the employment rate, earnings, and public benefit receipt of low-income adults in Washington State. The authors used a nonexperimental design to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefit receipt between those who took part in the WIA Adult Program and those who registered for employment services at the state Labor Exchange. The study found that, compared with those who registered for services at the Labor Exchange, participants in the WIA Adult Program had higher employment, earnings, and public benefit receipt. The quality of causal evidence presented in this report is low because the authors compared the treatment and comparison groups at different follow-up points and the groups were therefore not equivalent. This means we are not confident that the estimated effects are attributable to the WIA Adult Program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Workforce Investment Act (WIA) Adult Program","Adult and Dislocated Worker programs Job search assistance and supportive services Labor Exchange Other employment and reemployment Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Other training and education","Unemployed, Low income",,"United States",2006,http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=up_technica…,"Apprenticeship and Work-Based Training, Job Search Assistance Review Protocol" "Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments.","Henggeler, S., Halliday-Boykins, C., Cunningham, P., Randall, J., Shapiro, S., & Chapman, J. (2006). Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments. Journal of Consulting and Clinical Psychology, 74(1), 42-54.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Health and safety-Mod/high-Favorable impacts Health and safety","Summary: The study’s objective was to examine the marginal impacts of three layered treatments—drug court hearings, substance abuse therapy, and contingency management counseling—on criminal justice outcomes and positive drug screens among youth in Charleston, South Carolina, diagnosed with substance abuse or dependence. The authors randomly assigned eligible youth to one of three treatment groups—drug court, drug court with multisystemic therapy, or drug court with multisystemic therapy enhanced by contingency management—or to a control group that received typical family court services. The study found no statistically significant differences between the groups on measures of recidivism after one year. However, the study found that participants in the treatment groups reported committing fewer delinquent acts than those in the control groups. Effects on drug screens were mixed. The quality of causal evidence presented in this report is high for recidivism outcomes because they are based on a well-implemented randomized controlled trial but moderate for self-reported delinquency and drug screen outcomes because they had high attrition and the authors demonstrated that the groups being compared had similar baseline characteristics.","the Treatments","Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2006,,"Opportunities for Youth Review Protocol" "Employee development, commitment, and intention to turnover: A test of “employability” policies in action","Benson, G.S. (2006). Employee development, commitment, and intention to turnover: A test of “employability” policies in action. Human Resource Management Journal, 16(2), 173–192.","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the relationship between employees’ intent to leave (that is, employees’ stated intentions to leave the firm in the next year) and their participation in on-the-job training and company-sponsored training classes. The author used a statistical model to examine the association between employees’ participation in on-the-job and company-sponsored training classes and their intention to leave the firm in the next year, using data from an employee survey. The study found that participation in on-the-job training was associated with a reduction in employees’ intent to leave, but there was no significant relationship between participation in company sponsored training and intent to turnover. The quality of causal evidence presented in this report is low because the author did not ensure that groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the work-based training; other factors are likely to have contributed.","the on-the-job training","Work based and other occupational training","Employed, STEM professional","Professional, scientific, and technical services","United States",2006,https://doi.org/10.1111/j.1748-8583.2006.00011.x,"Employment and Training Review Protocol" "Workforce investment act services: Effect on dislocated worker reemployment (Doctoral Disertation)","Walker, M. A. (2006). Workforce investment act services: Effect on dislocated worker reemployment (Doctoral dissertation). Available from ProQuest Dissertations & Theses. Accession No. UMI 3244857. [Study 2: Contrast: WIA employment and training services vs. no service]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the effects of employment and training services offered through the Workforce Investment Act of 1998 (WIA) on employment and earnings outcomes. The author investigated similar research questions in another study, the profile of which can be found [here]. The other study examined the effects of WIA’s employment services on the same employment and earnings outcomes when there were no training services. The author used a statistical model in a nonexperimental analysis to compare outcomes of workers who lost their jobs for reasons including being laid off and either did or did not receive WIA employment services. The author used data from the state Employment Commission dislocated worker database, Workforce Investment Act Title IB Standardized Record Data. The study found a statistically significant, negative relationship between the WIA employment and training services and employment. Those receiving employment and training services had longer durations of unemployment than those who did not, by about 23 weeks. There were no statistically significant relationships between the WIA employment and training services and earnings. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated relationships are attributable to the WIA employment and training services; other factors are likely to have contributed to those relationships.","the Workforce Investment Act (WIA)","Adult and Dislocated Worker programs Job search assistance and supportive services Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Dislocated or displaced worker",,"United States",2006,https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1196&context=efl_etds,"Employment and Training Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington State. Upjohn Institute technical report no. TR06-020). [Private career school programs]","Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [Private career school programs]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Private Career School program on the employment rate, earnings, and public benefit receipt of adults with a high school education. The authors used a nonexperimental method to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefits of those who took part in the Private Career School program relative to those who registered for services at the Labor Exchange. The study found that, compared with those who registered for Labor Exchange services, Private Career School program participants had higher employment and earnings, and lower receipt of several types of public benefits. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the Private Career School program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Private Career School Programs","Other training and education","Adult, Unemployed","Other services, Transportation and warehousing","United States",2006,http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=up_technica…,"Employment and Training Review Protocol" "New evidence on earnings and benefit claims following the changes in the retirement earnings test in 2000.","Song, J. G., & Manchester, J. (2006). New evidence on earnings and benefit claims following the changes in the retirement earnings test in 2000. (ORES Working Paper 107). Washington, DC: Social Security Administration, Office of Policy, Office of Research, Evaluation, and Statistics. Also available in Journal of Public Economics, 91(3–4), 669–700 (2007).","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Low-No impacts Employment Public benefits receipt-Mod/high-Unfavorable impacts Public benefit receipt","Summary: The study examined the impact of the Senior Citizens Freedom to Work Act of 2000 on employment, earnings, and public benefit receipt outcomes of workers ages 65 to 69. The study was a nonexperimental analysis and used Social Security Administrative data to estimate the impacts. The study found that the Senior Citizens Freedom to Work Act of 2000 was associated with significant increases in the rate of Social Security claims for both sets of individuals affected by the law. The study found no statistically significant relationship between the Senior Citizens Freedom to Work Act of 2000 and earnings. The quality of casual evidence presented in this report for Social Security claims and earnings outcomes is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000, but other factors might also have contributed. The quality of causal evidence presented in this report on employment is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Senior Citizens Freedom to Work Act of 2000; other factors are likely to have contributed.","Senior Citizens Freedom to Work Act of 2000","Older workers' programs Other disparities or discrimination in employment and earnings","Adult, Older worker",,"United States",2006,,"Employment and Training Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). [Comm. and Tech. ABE]","Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [Comm. and Tech. ABE]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Mixed impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Community and Technical Colleges Adult Basic Education (ABE) program on the employment, earnings, and public benefits receipt of low-income adults. The authors used a nonexperimental method to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefits receipt between those who took part in the ABE program relative to those who registered for services at the Labor Exchange. The study found that, compared with those who registered for Labor Exchange services, ABE program participants had higher employment and earnings, and lower public benefits receipt. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the ABE program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","the Community and Technical Colleges Adult Basic Education (ABE) Program","Other training and education","Adult, Unemployed",,"United States",2006,http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=up_technica…,"Employment and Training Review Protocol" "Recidivism findings for the Juvenile Rehabilitation Administration’s mentoring program: Final report","Drake, E. (2006). Recidivism findings for the Juvenile Rehabilitation Administration’s mentoring program: Final report. Olympia, WA: Washington State Institute for Public Policy.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of the Washington State Juvenile Rehabilitation Administration’s mentoring program on recidivism. The author estimated the effect of the mentoring program using administrative data to compare outcomes among youth released from a juvenile rehabilitation facility who applied to participate in the program with the outcomes of similar youth who did not apply. The study did not find any statistically significant effects of the mentoring program on recidivism. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that any estimated effects would be attributable to the Washington State Juvenile Rehabilitation Administration’s mentoring program, but other factors might also have contributed. However, the study did not find statistically significant effects.","the Juvenile Rehabilitation Administration’s Mentoring Program","Mentoring Youth programs","Youth, Justice-involved, Disconnected youth",,"United States",2006,http://www.wsipp.wa.gov/ReportFile/947/Wsipp_Recidivism-Findings-for-the-Juveni…,"Opportunities for Youth Review Protocol" "Managing customers’ training choices: Findings from the Individual Training Account experiment","McConnell, S., Stuart, E., Fortson, K., Decker, P., Perez-Johnson, I., Harris, B., & Salzman, J. (2006). Managing customers’ training choices: Findings from the Individual Training Account experiment. Final report, Series: ETAOP 2007-01. Washington, DC: Employment and Training Administration, U.S. Department of Labor.","Low-Income Adults","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt Training-Mod/high-Favorable impacts Training","Summary: The study’s objective was to compare the impacts of individual training account (ITA) approaches on customers’ training completion, employment, earnings, and benefits receipt outcomes after 15 months. The authors randomly assigned about 8,000 adults and dislocated workers from eight workforce investment agencies across the United States to three ITA approaches: structured customer choice, maximum customer choice, or guided customer choice. The authors analyzed data from Unemployment Insurance (UI) records and from a survey conducted on a random subsample of study participants. The study found the customers in the structured customer choice approach received significantly higher amounts of public assistance benefits than did customers in the guided customer choice group. The study also found that the structured customer choice group had higher earnings and lower education and training compared with the maximum customer choice group, whereas the maximum customer choice group had lower earnings and higher public assistance benefits compared with the guided customer choice group. The quality of causal evidence provided in this study is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to differences in the ITA approaches, and not to other factors.","the Individual Training Accounts (ITA) Approaches","Other employment and reemployment Other training and education","Dislocated or displaced worker, Low income",,"United States",2006,http://scholar.google.com/scholar_url?url=http%3A//cire.mathematica-mpr.com/~/m…,"Employment and Training Review Protocol" "Workforce investment act services: Effect on dislocated worker reemployment (Doctoral Disertation)","Walker, M. A. (2006). Workforce investment act services: Effect on dislocated worker reemployment (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses. Accession No. UMI 3244857. [Study 1: Contrast: WIA employment services vs. no service]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-Unfavorable impacts Employment","Summary: The study’s objective was to examine the effects of employment services offered through the Workforce Investment Act of 1998 (WIA) on employment and earnings outcomes. The author investigated similar research questions in another study, whose profile can be found [here]. The other study examined the effects of WIA’s employment and training services on employment and earnings. The author used a statistical model in a nonexperimental analysis to compare the outcomes of workers who lost their jobs for reasons including being laid off and either did or did not receive WIA employment services. The author used data from the state Employment Commission’s dislocated worker database, Workforce Investment Act Title IB Standardized Record Data. The study found a statistically significant, negative relationship between the WIA employment services and employment. Those receiving employment services had longer durations of unemployment than those who did not, by about 21 weeks. There were no statistically significant relationships between the WIA employment services and earnings. The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated relationships are attributable to the WIA employment services; other factors are likely to have contributed to those relationships.","the Workforce Investment Act (WIA)","Adult and Dislocated Worker programs Job search assistance and supportive services Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA)","Dislocated or displaced worker",,"United States",2006,https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1196&context=efl_etds,"Employment and Training Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). [Community and Technical College Worker Retraining Program]","Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. [Community and Technical College Worker Retraining Program]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Mixed impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Community and Technical College (CTC) Worker Retraining program on the employment rate, earnings, and public benefit receipt of long-term unemployed and dislocated workers in Washington State. The authors used a nonexperimental design to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefit receipt between those who took part in the CTC Worker Retraining program and those who registered for employment services at the state Labor Exchange. The study found that, compared with those in the comparison group, participants in the CTC Worker Retraining program had higher employment and earnings, and lower public benefits receipt. The quality of causal evidence presented in this report is low because the authors compared the treatment and comparison groups at different follow-up points and the groups were therefore not equivalent. This means we are not confident that the estimated effects are attributable to CTC Worker Retraining program; other factors are likely to have contributed. This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.","Community and Technical College Worker Retraining (CTC WR) Program","Community college education and other classroom training Other training and education","Dislocated or displaced worker, Unemployed",,"United States",2006,http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=up_technica…,"Employment and Training Review Protocol" "Harford County juvenile drug court performance evaluation: Final report","Crumpton, D., Carey, S., Mackin, J., Finigan, M., Pukstas, K., Weller, J., and Brekhus, J. (2006). Harford County juvenile drug court performance evaluation: Final report. Portland, Oregon. NPC Research, 1-104.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to examine the impact of the Harford County (Maryland) Juvenile Drug Court on recidivism. The authors used administrative records to estimate the impact of the drug court by comparing the outcomes of youth who participated in the drug court with those of similar youth who did not. The study found that drug court participants had significantly fewer rearrests (36 percent fewer) and fewer days (59 percent fewer) on probation than youth who were eligible for the program but did not participate. Participants spent fewer days in secure and community detention in the first year after program entry than youth in the comparison group. However, the study found no significant effects on the number of adjudication hearings, days in residential treatment, or days in a group home. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Harford County Juvenile Drug Court, but other factors might also have contributed.","Harford County Juvenile Drug Court","Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2006,http://npcresearch.com/wp-content/uploads/HCJDC_Process_Outcome-Cost-FINAL-repo…,"Opportunities for Youth Review Protocol" "Regulatory federalism and workplace safety: Evidence from OSHA enforcement, 1981–1995","Bradbury, J. (2006). Regulatory federalism and workplace safety: Evidence from OSHA enforcement, 1981–1995. Journal of Regulatory Economics, 29(2), 211–224.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine if workplace fatalities were lower in states with federal or state enforcement of Occupational Safety & Health Administration (OSHA) regulations. The study used a regression model to compare changes over time in the workplace fatalities of states with federal or state enforcement of OSHA regulations. The study found that states with self-enforcement of OSHA regulations had lower fatality rates relative to states with federal OSHA enforcement; this difference was statistically significant. The quality of evidence presented in this study is low. This means we are not confident that the differences in workplace fatalities between states with self-enforcement and those with federal enforcement of OSHA regulations are attributable to the method of enforcement.","OSHA Enforcement Activities","Other health and safety Workplace injury",Employed,,"United States",2006,,"OSHA Enforcement Review Protocol" "Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). [WIA Dislocated Workers]","Hollenbeck, K., & Huang, W-J. (2006). Net impact and benefit-cost estimates of the workforce development system in Washington State. (Upjohn Institute technical report no. TR06-020). Kalamazoo, MI: W. E. Upjohn Institute for Employment Research. [WIA Dislocated Workers]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Mixed impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Dislocated Worker Program on the employment rate, earnings, and public benefit receipt of dislocated workers in Washington State. The authors used a nonexperimental design to compare the short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and public benefit receipt between those who took part in the WIA Dislocated Worker Program and those who registered for employment services at the state Labor Exchange. The study found that, compared with those who registered for services at the Labor Exchange, participants in the WIA Dislocated Worker Program had higher employment and earnings, although the findings for public benefit receipt were mixed. The quality of causal evidence presented in this report is low because the authors compared the treatment and comparison groups at different follow-up points and the groups were therefore not equivalent. This means we are not confident that the estimated effects are attributable to WIA Dislocated Worker Program; other factors are likely to have contributed.","the Workforce Investment Act (WIA) Dislocated Worker Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Adult, Dislocated or displaced worker",,"United States",2006,http://research.upjohn.org/cgi/viewcontent.cgi?article=1023&context=up_technica…,"Employment and Training Review Protocol" "Evaluation of the strengthening the connections between unemployment insurance and the One-Stop Delivery Systems Demonstration Project in Wisconsin","Almandsmith, S., Ortiz Adams, L., & Bos, H. (2006). Evaluation of the strengthening the connections between unemployment insurance and the One-Stop Delivery Systems Demonstration Project in Wisconsin. Oakland, CA: Berkeley Policy Associates.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-No impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study examined the impact of the Worker Profiling Reemployment Services program for Unemployment Insurance (UI) claimants in Wisconsin on several outcomes: the rate at which the UI claimants went back to work, average earnings, and UI benefit receipt.  The authors used a matching procedure and several statistical models to compare the outcomes of demonstration and comparison group members, using administrative data.  The study found that demonstration participants had higher average quarterly earnings, shorter durations of UI benefits, smaller proportions of maximum UI benefits paid, and a lower average total payout of UI benefits. The study found no statistically significant relationship between demonstration participation and employment outcomes.  The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Worker Profiling Reemployment Services program; other factors are likely to have contributed.","JSA services","RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2006,https://wdr.doleta.gov/research/FullText_Documents/Evaluation%20of%20the%20Stre…,"Reemployment Review Protocol" "Implementation and outcome evaluation of the Intensive Aftercare Program: Final report","Wiebush, R., Wagner, D., McNulty, B., Wang, Y., & Le, T. (2005). Implementation and outcome evaluation of the Intensive Aftercare Program: Final report. National Council on Crime and Delinquency. Washington, DC: U.S. Department of Justice, Office of Justice Programs.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence",,"Summary: The study’s objective was to measure the impact of participation in an Intensive Aftercare Program (IAP) for high-risk youth from juvenile justice facility placements on recidivism. The authors used a random assignment design and multivariate regression analysis to estimate impacts on aggregate recidivism scores, the sum of all subsequent offenses, weighted by severity. Data for the study were collected from standardized forms, surveys, risk assessment tests, and state agency and police records. The study found no statistically significant effects of the program on participants’ aggregate recidivism scores. The quality of causal evidence presented in this report is moderate for some outcomes and low for other outcomes. This means we have little confidence that any estimated effects would be attributable to the IAP. However, the study found few statistically significant effects.","the Intensive Aftercare Program (IAP)","Reentry Youth programs","Youth, Justice-involved, Disconnected youth, Male",,"United States",2005,http://files.eric.ed.gov/fulltext/ED484683.pdf,"Opportunities for Youth Review Protocol" "Are empowerment and education enough? Underdiversification in 401(k) plans","Choi, J., Laibson, D., & Madrian, B. (2005). Are empowerment and education enough? Underdiversification in 401(k) plans. Brookings Papers on Economic Activity 2005, (2), 151-213.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-No impacts Employer benefits receipt","Summary: The study examined factors that can affect diversification from employer stock in 401(k) plans. The study analyzed the impact of loosened diversification restrictions and media coverage of highly publicized events on 401(k) holdings in employer stock. The authors assessed factors affecting diversification in 401(k) plans through four studies. The first was an interrupted time series analysis of employer stock holdings at an unnamed company. The other three studies explored the impact of media attention to the Enron, WorldCom, and Global Crossing bankruptcies through regression analysis. All analyses were based on transaction-level and annual data from 1997 to 2003 from Hewitt Associates, a benefits administration company. The authors attributed only a 2.4 percentage point decline in employer stock holdings to news coverage of the Enron, WorldCom, and Global Crossing bankruptcies across the entire Hewitt 401(k) Index. The quality of causal evidence presented in this report is low. In the first study, the authors did not account for prior trends in the outcome variable and in the second through fourth studies they did not provide information a reader would need to assess changes in sample composition. This low rating means we are not confident that the estimated effects are attributable to any of the interventions studied, whether loosened diversification restrictions or media coverage of the Enron, Global Crossing, and WorldCom bankruptcies. Other factors are likely to have contributed.","Factors Promoting 401(k) Diversification","Employer provided retirement benefits",Employed,,"United States",2005,,"Behavioral Finance: Retirement Review Protocol" "The declining effects of OSHA inspections on manufacturing injuries, 1979-1998","Gray, W., & Mendeloff, J. (2005). The declining effects of OSHA inspections on manufacturing injuries, 1979-1998. Industrial and Labor Relations Review, 58(4), 571-587.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of federal OSHA inspections in manufacturing establishments on workplace injuries during three separate time periods: 1979–1985, 1987–1991, and 1992–1998. The study used a regression model to compare the changes in workplace injuries in manufacturing establishments that had received an inspection within the past four years to those in establishments that had not received inspections. The study found that OSHA inspections were associated with statistically significant reductions in lost-workday injuries of 10 percent in 1979–1985 and 4 percent in 1987–1991, but had no statistically significant effect on these injuries in 1992–1998. The quality of causal evidence presented in this study is low. This means we are not confident that the differences in workplace injuries between firms that did and did not receive OSHA inspections are attributable to the inspections.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,Manufacturing,"United States",2005,http://digitalcommons.ilr.cornell.edu/cgi/viewcontent.cgi?article=1140&context=…,"OSHA Enforcement Review Protocol" "Net impact estimates for services provided through the Workforce Investment Act. ETA Occasional Paper 2005-06. [WIA Dislocated Worker]","Hollenbeck, K., Schroeder, D., King, C., & Huang, W. (2005). Net impact estimates for services provided through the Workforce Investment Act. ETA Occasional Paper 2005-06. Washington, DC: U.S. Department of Labor, Employment and Training Administration. [WIA Dislocated Worker]","Apprenticeship and Work-Based Training","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Workforce Investment Act (WIA) Dislocated Worker Program’s core, intensive, and training services on the employment rate, earnings, and benefits receipt of low-income dislocated workers ages 22 to 64. The authors established three treatment groups based on the level of WIA services used and matched them to three comparison groups that participated in Employment Services (ES) and/or the core WIA services. The study found that the employment rate and average quarterly earnings were significantly higher for those in each of the treatment groups compared with their matched comparison groups. Benefit receipt was also significantly lower for those in the treatment groups than in their comparison groups. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the WIA Dislocated Worker Program; other factors are likely to have contributed.","the Workforce Investment Act (WIA) Dislocated Worker Program","Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) Workforce Investment Act (WIA)/Workforce Innovation and Opportunity Act (WIOA) training programs","Dislocated or displaced worker",,"United States",2005,https://wdr.doleta.gov/research/FullText_Documents/Net%20Impact%20Estimates%20f…,"Employment and Training Review Protocol" "The effect of the SSI program on labor supply: Improved evidence from Social Security administrative files","Neumark, D., & Powers, E. T. (2005). The effect of the SSI program on labor supply: Improved evidence from Social Security administrative files. Social Security Bulletin, 65(3), 45-60.","Older Workers","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment","Summary: The study examined the relationship between generous state Supplemental Security Income (SSI) benefits and the employment of older worker nearing SSI eligibility age. The authors used a regression model and data from the Survey of Income and Program Participants (SIPP) and Social Security Administration (SSA) to estimate the effect. The study found that older males who were likely SSI participants in generous states worked significantly fewer hours and had lower employment rates compared with those living in less generous states. The authors, using the SSA administrative records, also found that likely older participants (ages 60 to 64) worked significantly fewer hours than younger ones. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to generosity of state SSI benefits; other factors are likely to have contributed.","state Supplemental Security Income (SSI) benefits","Older workers' programs Other employment and reemployment","Adult, Older worker",,"United States",2005,https://www.ssa.gov/policy/docs/ssb/v65n3/v65n3p45.html,"Employment and Training Review Protocol" "Cost-effectiveness of targeted reemployment bonuses","O’Leary, C.J., Decker, P.T., Wandner, S.A. (2005). Cost-effectiveness of targeted reemployment bonuses. Journal of Human Resources, 40, 270-279.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study examined the impact of the Pennsylvania Reemployment Bonus Demonstration on the amount of Unemployment Insurance (UI) receipt and on earnings of UI claimants approximately one year after program enrollment, both overall and for subgroups of claimants defined by their likelihood of exhausting UI benefits. The authors also examined the impact of the Washington Reemployment Bonus Demonstration; this study is an additional source to the main source examining that intervention, profiled here. About 15,000 eligible claimants were randomly assigned to one of six treatment groups or a control group. The six treatment groups were offered different combinations of reemployment bonus amounts and periods within which they were required to find a job in order to claim the bonus. The authors analyzed state administrative data and survey data of a subsample of approximately 5,000 participants from the original demonstration. The study found that the pooled bonus treatments in Pennsylvania reduced the total UI benefit amount received, but did not have a significant impact on earnings. Pooling across all bonus treatments, Pennsylvania treatment group members received $113 less in UI benefits, on average, than their control group counterparts. The quality of causal evidence for the Pennsylvania demonstration is low because the authors did not demonstrate that they adjusted for changes in rates of assignment to treatment and control conditions and did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Pennsylvania Reemployment Bonus Demonstration; other factors are likely to have contributed. See more CLEAR profiles related to The Reemployment Bonus Experiments.","Pennsylvania Reemployment Bonus Demonstration","Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2005,,"Reemployment Review Protocol" "Evaluating an experimental intensive juvenile probation program: Supervision and official outcomes","Lane, J., Turner, S., Fain, T., & Sehgal, A. (2005). Evaluating an experimental intensive juvenile probation program: Supervision and official outcomes. Crime & Delinquency, 51(1), 26-52.","Opportunities for Youth","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence",,"Summary: The study’s objective was to examine the impact of the South Oxnard Challenge Project (SOCP), an alternative juvenile probation program, on recidivism outcomes. The study was a randomized controlled trial. The authors used administrative data from the program and probation records to compare the outcomes of youth randomly assigned to the SOCP with those of youth randomly assigned to a control group that received standard probation services. The study found no statistically significant effects of SOCP on arrests or referrals to probation, convictions, or incarceration. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects would be attributable to SOCP and not to other factors. However, the study did not find statistically significant effects.","South Oxnard Challenge Project (SOCP)","Substance abuse recovery Youth programs Behavioral Interventions","Youth, Justice-involved, Disconnected youth",,"United States",2005,,"Opportunities for Youth Review Protocol" "Inside the black box: How do OSHA inspections lead to reductions in workplace injuries?","Mendeloff, J., & Gray, W. (2005). Inside the black box: How do OSHA inspections lead to reductions in workplace injuries? Law and Policy, 27(2), 219-237.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effects of OSHA inspections that resulted in penalties or citations on the types of injuries workers suffered. The study used a regression model to compare the changes in workplace injuries of manufacturing firms that had received an inspection that resulted in a penalty or citation to those that had not received a penalty or citation. The group of firms that had not received a penalty or citation combined two types of firms: firms that had not been inspected and firms that received inspections that did not result in a penalty or citation. The study found that OSHA inspections that resulted in a penalty were associated with statistically significant decreases in lost-workday injuries. Among the specific citations studied, citations for the standards for general requirements for personal protective equipment and general machine guarding were associated with statistically significant reductions in some types of lost-workday injuries. The quality of the causal evidence presented in this study is low. This means we are not confident that OSHA inspections that resulted in penalties or citations caused the reductions in workplace injuries.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,Manufacturing,"United States",2005,,"OSHA Enforcement Review Protocol" "Save More Tomorrow™: Using behavioral economics to increase employee saving","Thaler, R., & Benartzi, S. (2004). Save More Tomorrow™: Using behavioral economics to increase employee saving. Journal of Political Economy, 112(S1), S164-S187.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study aimed to determine whether a retirement savings plan that automatically increased 401(k) contributions whenever an enrolled employee’s salary increased improved savings rates among employees. The authors compared savings rates among employees who opted to participate in the plan with savings rates among employees who declined to join the plan. Across the three experimental sites, average savings rates tended to increase, with higher increases for program participants than for nonparticipants. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that the difference in savings rates between program participants and nonparticipants results from participation in the savings plan.","Save More Tomorrow™","Employer provided retirement benefits",Employed,,"United States",2004,,"Behavioral Finance: Retirement Review Protocol" "Saving for retirement on the path of least resistance","Choi, J., Laibson, D., Madrian, B., and Metrick, A. (2004). Saving for retirement on the path of least resistance. Working paper. Cambridge, MA.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study examined the impact of different 401(k) plan designs on employees’ saving behavior at several U.S.-based firms. The analysis was based on administrative data provided by each company. The study used five analyses—one based on an interrupted time series (ITS) design, one based on a randomized controlled trial (RCT), and three quasi-experimental regression analyses—to estimate impacts on 401(k) participation and contribution rates. The study found that employees’ decision making was influenced by defaults, such as automatic enrollment and automatic disbursements from 401(k) plans. It also found that the gap in 401(k) participation rates between new employees facing a mandatory waiting period for eligibility and those with immediate eligibility disappeared shortly after the former group gained eligibility. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the various 401(k) program designs evaluated in this study. Other factors are likely to have contributed.","401(k) Plan Design Changes","Employer provided retirement benefits",Employed,,"United States",2004,http://www.hks.harvard.edu/fs/bmadria/Documents/Madrian%20Papers/Saving%20for%2…,"Behavioral Finance: Retirement Review Protocol" "Long-term ethylene oxide exposure trends in U.S. hospitals: relationship with OSHA regulatory and enforcement actions","LaMontagne, A., Oakes, J., & Turley, R. (2004). Long-term ethylene oxide exposure trends in U.S. hospitals: Relationship with OSHA regulatory and enforcement actions. American Journal of Public Health, 94(9), 1614-1619.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Mixed impacts Health and safety","Summary: The study’s objective was to examine differences in hospital worker exposures to ethylene oxide (EtO) after the implementation of the 1984 and 1988 OSHA EtO standards. The authors used a random effects model to examine changes over time in hospital worker exposures to EtO. The study reported a steady, statistically significant decline in EtO exposures for the first several years after each OSHA standard was set. The quality of evidence presented in this study is low. This means we are not confident that changes in worker EtO exposures over time are attributable to OSHA’s implementation of EtO standards.","OSHA Enforcement Activities","Other health and safety Safety",Employed,"Health care and social assistance","United States",2004,http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1448503/pdf/0941614.pdf,"OSHA Enforcement Review Protocol" "Evaluation of OSHA’s impact on workplace injuries and illnesses in manufacturing using establishment-specific targeting of interventions: programmed inspections vs. high hazard notification letters","ERG. (2004). Evaluation of OSHA’s impact on workplace injuries and illnesses in manufacturing using establishment-specific targeting of interventions: programmed inspections vs. high hazard notification letters. Final report. Lexington, MA: Author.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine the impact of two types of notifications sent by the Occupational Safety & Health Administration (OSHA)—those with and without subsequent inspections—on workplace injuries. The study used a regression model to analyze annual changes in workplace injuries in manufacturing firms that received OSHA notifications of high existing injury rates and/or inspections in the current or past two years. The study found that firms that received notifications without a subsequent inspection experienced a statistically significant decline in injury rates. Firms that received a notification and an inspection experienced larger declines. The quality of the causal evidence presented in this study is low. This means we are not confident that OSHA notifications and inspections caused the reductions in workplace injuries.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,,"United States",2004,,"OSHA Enforcement Review Protocol" "For better or for worse default effects and 401(k) savings behavior","Choi, J.J., Laibson, D., Madrian, B.C., & Metrick, A. (2004). For better or for worse default effects and 401(k) savings behavior. National Bureau of Economic Research, 81-126","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s key objective was to examine the impact of automatic 401(k) plan enrollment on enrollment rates and contribution levels at three firms in the United States. The analysis was based on data on enrollment in and contributions to retirement plans from the companies themselves. The study used an interrupted time series (ITS) design, analyzing changes in outcomes before and after the implementation of automatic enrollment. The study found that automatic enrollment increased participation rates. In addition, automatic enrollment led to a larger percentage of participants investing at the plan’s default contribution rate. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the companies’ automatic enrollment policies. Other factors are likely to have contributed.","Automatic Enrollment","Employer provided retirement benefits",Employed,,"United States",2004,,"Behavioral Finance: Retirement Review Protocol" "Estimating the returns to community college schooling for displaced workers","Jacobson, L., LaLonde, R., & Sullivan, D. (2004). Estimating the returns to community college schooling for displaced workers. (IZA discussion paper no. 1018). Bonn, Germany: Institute for the Study of Labor.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of taking community college courses on the earnings of displaced workers in Washington State. The authors matched data from unemployment insurance wage records to community college records and used regression models to compare the outcomes of displaced workers who earned community college credits with the outcomes of those who did not. The study found that long-term earnings among displaced workers increased 9 percent for men and 13 percent for women as a result of one year of community college, and that the effects were largest for those who took quantitative or vocational courses. The quality of causal evidence provided in this study is moderate. This means we are somewhat confident that the estimated effects are attributable to the community college course taking, but other factors might also have contributed.","the Study","Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2004,http://repec.iza.org/dp1017.pdf,"Community College Review Protocol" "Workplace Health and Safety Regulations: Impact of Enforcement and Consultation on Workers’ Compensation Claims Rates in Washington State","Baggs, J., Silverstein, B., & Foley, M. (2003). Workplace health and safety regulations: Impact of enforcement and consultation on workers’ compensation claims rates in Washington State. American Journal of Industrial Medicine, 43(5), 483-494.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study examined the effects of Washington’s Occupational Safety & Health Administration (OSHA) State Plan (named WISHA) inspections and consultations on workers’ compensation claim rates for employers in Washington State. The study used a difference-in-differences model to compare the changes in compensable claim rates for employers that had received WISHA inspections or consultations in the previous year with those that had not received the specific WISHA intervention. The study found that WISHA enforcement activity was associated with a statistically significant reduction in workers’ compensation claims rates for fixed-site industries (for example, manufacturing) but not nonfixed-site industries (for example, construction). The quality of the causal evidence presented in this study is low. This means we are not confident that the relative reductions in the compensable claim rates between employers that experienced WISHA enforcement activities or consultations and employers that did not were attributable to the WISHA enforcement activities or consultations.","OSHA Enforcement Activities","Inspections Other health and safety Workers compensation Workplace injury",Employed,,"United States",2003,http://onlinelibrary.wiley.com/doi/10.1002/ajim.10209/abstract,"OSHA Enforcement Review Protocol" "Is the threat of reemployment services more effective than the services themselves? Evidence from random assignments in the UI system","Black, D., Smith, J., Berger, M., & Noel, B. (2003). Is the threat of reemployment services more effective than the services themselves? Evidence from random assignments in the UI system. American Economic Review, 93(4), 1313-1327.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study examined the impact of the Kentucky Worker Profiling and Reemployment Services (WPRS) on Unemployment Insurance (UI) benefit receipt and earnings.  In this study, about 2,000 UI claimants in Kentucky were randomly assigned to the treatment group, which was required to participate in reemployment services, or to the control group, which was not required to participate but could receive services voluntarily. The authors collected administrative data on UI benefit receipt and earnings for all study participants. The study found that those in the WPRS treatment group had a statistically significant reduction in UI benefit receipt of 2.2 weeks during the six-quarter follow-up period, compared to the control group. There were no statistically significant impacts of the program on the fraction of recipients exhausting benefits or total UI benefits received. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial (RCT). This means we are confident that the estimated effects are attributable to the WPRS and not to other factors.",Profiling,"RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2003,,"Reemployment Review Protocol" "How much choice is too much? Contributions to 401 (k) retirement plans","Iyengar, S.S., Huberman, G., & Jiang, W. (2003). How much choice is too much? Contributions to 401 (k) retirement plans. Pension Research Council working paper.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Unfavorable impacts Employer benefits receipt","Summary: This study’s objective was to examine whether the number of fund options offered by a 401(k) retirement plan influenced employees’ decisions to enroll in the plan. The authors analyzed employees’ participation rates in 401(k) plans using administrative data provided by an investment management company and regression analysis, controlling for both employee- and plan-level characteristics. The analysis demonstrated that as the number of fund options increased, employees became less likely to enroll in a 401(k) plan. Adding one fund to the list of investment options was associated with a decrease in participation of 0.15 to 0.20 percentage points. The quality of causal evidence presented in this study is low. This means that we are not confident that the increase in the number of funds offered caused the decrease in 401(k) enrollment. Other factors are likely to have contributed to the observed relationship.","the Number of Fund Choices","Employer provided retirement benefits",Employed,,"United States",2003,,"Behavioral Finance: Retirement Review Protocol" "The role of information and social interactions in retirement plan decisions: Evidence from a randomized experiment","Duflo, E., & Saez, E. (2003). The role of information and social interactions in retirement plan decisions: Evidence from a randomized experiment. Quarterly Journal of Economics, 118(3), 815–842.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employer benefits receipt-Mod/high-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine the impact of receiving information about tax-deferred retirement accounts (TDAs) on enrollment in the accounts. The study used a randomized controlled trial (RCT) at a large university. Departments within the university were randomly assigned into two groups. In the treatment group, nonfaculty employees were randomly selected to receive or not receive a letter offering a $20 incentive to attend an employee-benefits fair, at which information on TDAs was provided. In the other group of departments, no employees received a letter. The study examines both the effect of receiving the letter or being in a department in which some people received the letter. Under most model specifications, the study found that people in both treatment groups had higher enrollment in TDAs, as measured 4.5 and 11.0 months after the fair. The quality of the causal evidence presented in this study is high. This means we are confident that the estimated effects of the intervention are attributable to the intervention itself, and not some other factor.","Incentives for Attending a Benefits Fair","Employer provided retirement benefits",Employed,,"United States",2003,,"Behavioral Finance: Retirement Review Protocol" "Evaluation of the Significant Improvement Demonstration Grants for the provision of reemployment services for UI claimants","Needels, K., Corson, W., & Van Noy, M. (2002). Evaluation of the Significant Improvement Demonstration Grants for the provision of reemployment services for UI claimants. Princeton, NJ: Mathematica Policy Research.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-No impacts Public benefit receipt","Summary: The study examined the impacts of the Significant Improvement Demonstration Grants—awarded to 11 states to improve reemployment services for Unemployment Insurance (UI) claimants—on weeks of UI receipt and UI benefit exhaustion. For each state, the authors compared outcomes before and at the end of the grant period using monthly data that states reported to the UI Service. The study found no statistically significant relationships between the Significant Improvement Demonstration Grants and UI benefit receipt outcomes. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Significant Improvement Demonstration Grants; other factors are likely to have contributed.","Significant Improvement Demonstration Grants","Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",2002,https://www.doleta.gov/reports/searcheta/occ/papers/UI_final.pdf,"Reemployment Review Protocol" "The returns to community college schooling for displaced workers.","Jacobson, L., LaLonde, R., & Sullivan, D. G. (2001). The returns to community college schooling for displaced workers. Retrieved from: https://files.eric.ed.gov/fulltext/ED481841.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment","Summary: The study's objective was to examine the impact of community college schooling on displaced workers’ earnings and employment outcomes. This summary focuses on the Washington State sample. The study used a nonexperimental design to compare the earnings and employment outcomes between displaced workers who completed community college schooling and displaced workers who did not receive such schooling. Using state unemployment insurance earnings records and community college transcripts, the authors conducted statistical models to examine the differences in outcomes between the groups. The study found that displaced workers who completed at least one community college course earned more than displaced workers who either did not enroll in community college courses or who enrolled but did not complete any courses. The study also found community college schooling to be associated with higher hourly wages and number of hours worked for both males and females and higher employment rates for females only. However, the authors did not provide tests of statistical significance. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to enrollment in community college, but other factors might also have contributed.",,"Employment and Training Services Training and Education Basic skills Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2001,https://files.eric.ed.gov/fulltext/ED481841.pdf,"Community College Review Protocol" "The power of suggestion: Inertia in 401(k) participation and savings behavior","Madrian, B. C., & Shea, D. F. (2001). The power of suggestion: Inertia in 401(k) participation and savings behavior. The Quarterly Journal of Economics, 116(4), 1149-1187.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Mixed impacts Employer benefits receipt","Summary: The study examined the impact of two simultaneously implemented changes in the 401(k) plan of a large, Fortune 500 company: (1) automatic enrollment into the plan immediately after hire (a change from requiring employees to opt in to the plan to requiring them to opt out); and (2) removing the one-year tenure requirement for employees to participate in the 401(k) plan, enabling immediate contributions for employees with less than one year of tenure at the firm. The study used employee-level data from the company. It used an interrupted time series design to compare the participation and contribution rates of those hired before the company’s 401(k) plan changes to the behaviors of those hired after the changes were implemented. The authors compared employees subject to changes (1) and (2) to those subject to change (2) only and those subject to change (2) only to those subject to neither change. Employees subject to both immediate contributions and automatic enrollment had higher 401(k) participation rates and lower 401(k) contribution rates than those subject to immediate contributions only. Employees subject to immediate contributions had only slightly higher participation rates compared with those not subject to either plan change. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the company’s changes to its 401(k) plan design. Other factors are likely to have contributed.","Allowing Immediate 401(k) Contributions and Automatic Enrollment","Employer provided retirement benefits",Employed,"Large business","United States",2001,,"Behavioral Finance: Retirement Review Protocol" "The returns to community college schooling for displaced workers","Jacobson, L., LaLonde, R., & Sullivan, D. G. (2001). The returns to community college schooling for displaced workers. Retrieved from: https://files.eric.ed.gov/fulltext/ED481841.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages","Summary: The study's objective was to examine the impact of the Displaced Workers Educational Training Program (DWETP) on earnings. This summary focuses on the Pittsburgh sample. The study used a nonexperimental design to compare the outcomes of displaced workers who completed community college courses as part of the DWETP and displaced workers who did not receive such schooling. Using state unemployment insurance earnings records and community college transcripts, the authors conducted statistical models to examine the differences in earnings between the groups. The study found that post-schooling earnings for both male and female participants in the DWETP were greater than their counterparts in the comparison group who did not participate in the program. However, the authors did not provide tests of statistical significance. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to enrollment in the DWETP, but other factors might also have contributed.","the Displaced Workers Educational Training Program (DWETP)","Employment and Training Services Training and Education Basic skills Community college education and other classroom training","Adult, Dislocated or displaced worker",,"United States",2001,https://files.eric.ed.gov/fulltext/ED481841.pdf,"Community College Review Protocol" "Assessing OSHA performance: New evidence from the construction industry","Weil, D. (2001). Assessing OSHA performance: evidence from the construction industry. Journal of Policy Analysis and Management, 20(4), 651-74.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Compliance-Low-Favorable impacts Compliance","Summary: The study’s objective was to examine the effect of Occupational Safety & Health Administration (OSHA) inspections on compliance behavior among construction contractors from 1987 to 1993. Although OSHA no longer operates as it did during this period, this study provides interesting historical context. The study also contains a descriptive analysis that examined whether OSHA regulatory standards were related to causes of injuries and fatalities (analysis not included in this summary). The study used regression models to analyze the change in compliance probability between the first and subsequent inspections for a given contractor and for a given contractor at a specific work site. The study found that the probability of citation decreased with each additional inspection, with the largest difference between the first and second inspections. The quality of causal evidence presented in this study is low. Although the study provides interesting descriptive information, it does not provide evidence of a causal effect of reinspections on compliance behavior.","OSHA Enforcement Activities","Inspections Safety",Employed,Construction,"United States",2001,,"OSHA Enforcement Review Protocol" "Effect of state workplace safety laws on occupational injury rates","Smitha, M., Kirk, K., Oestenstad, K., Brown, K., & Lee, S. (2001). Effect of state workplace safety laws on occupational injury rates. Journal of Occupational Environmental Medicine, 43(12), 1001-1010.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of four types of mandatory state workplace safety interventions on occupational injury rates in manufacturing industries. The study used a regression model to compare differences in injury rates for industries in which higher or lower percentages of the workforce were affected by the four mandatory safety requirements. The study found that the mandatory safety committee requirements were associated with statistically significant decreases in injury rates; the other three initiatives were not. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in injury rates between industries in states with the mandatory safety laws and industries in states without these laws are attributable solely to the laws.","OSHA Enforcement Activities","Other health and safety Workplace injury",Employed,Manufacturing,"United States",2001,,"OSHA Enforcement Review Protocol" "Assisting Unemployment Insurance claimants: The long-term impacts of the Job Search Assistance Demonstration","Decker, P., Olsen, R., & Freeman, L. (2000). Assisting Unemployment Insurance claimants: The long-term impacts of the Job Search Assistance Demonstration. Princeton, NJ: Mathematica Policy Research.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Job Search Assistance Demonstration in Washington, D.C., and Florida on Unemployment Insurance (UI) benefits receipt, earnings, and employment. In this demonstration, about 20,000 UI claimants were randomized into one of three intervention groups, all of which received some variation of job search assistance services, or into a control group, which could receive existing services in the community. For all participants, the study team collected administrative data, which provided information on UI receipt, earnings, and employment. The study found that structured job search assistance was generally successful at reducing UI benefits receipt and increasing earnings in Washington, D.C., but not in Florida. The findings for individualized job search assistance treatment groups were less robust. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the job search assistance strategies tested, and not to other factors.","JSA services","Job search assistance and supportive services Other training and education","Dislocated or displaced worker, Unemployed",,"United States",2000,,"Job Search Assistance, Reemployment Review Protocol" "Two years after a job loss: Long-term impact of the JOBS Program on reemployment and mental health","Vinokur, A., Schul, Y., Vuori, J., & Price, R. (2000). Two years after a job loss: Long-term impact of the JOBS Program on reemployment and mental health. Journal of Occupational Health Psychology, 5(1), 32-47.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Employment-Low-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of the JOBS II program on long-term employment, earnings, and monthly hours worked. The authors randomly assigned eligible unemployed individuals to either a treatment group, which could participate in JOBS II, or the control group, which could receive other services in the community, but not JOBS II. The study included a pre-test two weeks before the intervention and follow-up questionnaires 2, 6, and 24 months after the intervention. The authors found that JOBS II was associated with positive effects on the treatment group’s rate of reemployment and monthly earnings as measured two years after assignment, compared with the control group. The quality of causal evidence presented in this report is low because the study was a randomized controlled trial with unknown attrition and lack of sufficient controls in the analysis. This means we are not confident that the estimated effects are attributable to the JOBS II program; other factors are likely to have contributed.","the JOBS II Program","Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",2000,,"Reemployment Review Protocol" "Measuring the effect of public labor exchange (PLX) referrals and placements in Washington and Oregon","Jacobson, L., & Petta, I. (2000). Measuring the effect of public labor exchange (PLX) referrals and placements in Washington and Oregon (Workforce Security Research Publications 2000–06). Washington, DC: U.S. Department of Labor.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Mixed impacts Public benefit receipt","Summary: The study examined the impact of receiving public labor exchange (PLX) job referrals and placements on the duration of Unemployment Insurance (UI) benefit receipt for UI recipients in Oregon and Washington. The authors used statistical models to compare two distinct treatment groups to a comparison group within each state. The treatment groups both received PLX job search assistance, a job referral, and a job offer; one group did not actually work at the position offered while the other did. The comparison group received job search assistance but not PLX job referrals or placements. The authors used data from the Oregon Employment Department and the Washington State Employment Security Department. The study found that individuals in the treatment group tended to receive UI benefits for fewer weeks than individuals in the comparison group; this was a statistically significant difference. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before the intervention. This means we are not confident that the estimated relationships are attributable to PLX job referrals or placements; other factors are likely to have contributed.",Other,"RESEA Job search assistance and supportive services Labor Exchange Wagner-Peyser Act",Unemployed,,"United States",2000,https://wdr.doleta.gov/owsdrr/00-6/00-6.pdf,"Reemployment Review Protocol" "Evaluation of Worker Profiling and Reemployment Services systems: Final Report.","Dickinson, K., Kreutzer, S., West, R., & Decker, P. (1999). Evaluation of Worker Profiling and Reemployment Services systems: Final report. Research and Evaluation Report Series 99-D. Washington, DC: U.S. Department of Labor, Employment, and Training Administration.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study assessed the effectiveness of Worker Profiling and Reemployment Services (WPRS), a system for predicting which Unemployment Insurance (UI) recipients are at greatest risk of exhausting benefits and offering them early intervention services.  The study used a nonexperimental design to compare the UI benefit receipt, employment, and earnings outcomes of WPRS participants in six states to the outcomes of similar UI claimants who did not receive WPRS. The main data source was state UI administrative records for nearly 457,000 UI claimants. The study found that WPRS reduced UI benefit weeks and amounts in three of the states during the benefit year. However, there were few statistically significant impacts on employment rates or earnings in the four quarters after the initial claim, and some of the impacts were in an unexpected direction. The quality of causal evidence presented in this report is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to WPRS, but other factors might also have contributed.",Profiling,"RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",1999,,"Reemployment Review Protocol" "The employment and training outcomes of a job training partnership act program in a community college setting","Carter, D. L. (1999). The employment and training outcomes of a job training partnership act program in a community college setting (Unpublished doctoral dissertation). University of Florida.","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study's objective was to examine the impact of a Job Training Partnership Act (JTPA) funded vocational training program at a community college in North Florida on earnings outcomes. The author used an interrupted time series design to compare outcomes of participants before and after they participated in the JTPA program. Data for the study were compiled from the community college's existing JTPA records. The study found that participation in JTPA funded training was significantly related to higher post-training wages. The quality of causal evidence presented in this report is low because the author did not observe outcomes for multiple periods before or after the program nor account for selection into the intervention. This means we are not confident that the estimated effects are attributable to the JTPA funded vocational training program; other factors are likely to have contributed.","Job Training Partnership Act","Employment and Training Services Training and Education Basic skills Capacity building programs Community college education and other classroom training","Youth, Adult, Other barriers, Dislocated or displaced worker, Low income",,"Rural, United States",1999,,"Community College Review Protocol" "Two essays on unemployment insurance: Claimant responses to policy changes. [Comparison between claimants who applied for UI benefits before and after WPRS]","Noel, B. J. (1998). Two essays on unemployment insurance: Claimant responses to policy changes (Doctoral dissertation). Retrieved from UMI (UMI No. 9922624). [Comparison between claimants who applied for UI benefits before and after WPRS]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of reemployment services through the Worker Profiling and Reemployment System (WPRS) on unemployment insurance (UI) benefit receipt and earnings for individuals who applied for UI in Kentucky. The authors investigated similar research questions for another contrast, the profile of which can be found here. The author compared outcomes for treatment group members, who were eligible to receive reemployment services, and comparison group members, who were not eligible to receive the reemployment services but could access other services in the community, using data from the Kentucky Department of Employment Services Unemployment Insurance administrative records. The study found statistically significant relationships between the WPRS reemployment services and UI benefit receipt and earnings, where those offered the reemployment services received fewer UI benefits in 26 weeks and earned more in one year, compared to those who did were not eligible to receive reemployment services. The quality of causal evidence presented in this report is low because the author did not account for trends in outcomes before the intervention. This means we are not confident that the estimated relationships are attributable to the WPRS reemployment services; other factors are likely to have contributed.  ",Profiling,"RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",1998,,"Reemployment Review Protocol" "Two essays on unemployment insurance: Claimant responses to policy changes [Contrast 1: Comparison between claimants who applied for UI benefits after WPRS only]","Noel, B. J. (1998). Two essays on unemployment insurance: Claimant responses to policy changes (Doctoral dissertation). Retrieved from UMI (UMI No. 9922624). [Comparison between claimants who applied for UI benefits after WPRS only]",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of reemployment services through the Worker Profiling and Reemployment System (WPRS) on unemployment insurance (UI) benefit receipt and earnings for individuals who applied for UI in Kentucky. The authors investigated a similar research questions in another contrast, the profiles of which can be found here. The author compared outcomes for treatment group members, who were eligible to receive reemployment services, and comparison group members, who were not eligible to receive the reemployment services but could access other services in the community, using data from the Kentucky Department of Employment Services Unemployment Insurance administrative records. The study found statistically significant relationships between reemployment services and UI benefit receipt, where those who were offered the reemployment services received less UI benefits in 26 weeks, compared to those who did not receive reemployment services. There were no statistically significant relationships between reemployment services and earnings. The quality of causal evidence presented in this report is low because the author did not ensure that the groups compared were similar before the intervention. This means we are not confident that the estimated relationships are attributable to the WPRS reemployment services; other factors are likely to have contributed.",Profiling,"RESEA Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",1998,,"Reemployment Review Protocol" "Youth Corps: Promising strategies for young people and their communities","Jastrzab, J., Blomquist, J., Masker, J., & Orr, L. (1997). Youth Corps: Promising strategies for young people and their communities. Cambridge, MA: Abt Associates Inc.","Opportunities for Youth","Study Type: Descriptive Analysis",,,"Summary: This report provides a high-level summary of a study on Youth Corps programs, which have engaged disadvantaged youth in community service projects since 1933. An in-depth impact analysis is contained in Jastrzab et al. 1996. The study’s objective was to gain insight into participants’ contributions and experiences; analyze program impacts on educational, employment, and behavioral outcomes; and assess the costs and benefits of the programs. The authors analyzed data on initiatives sponsored by the Commission on National and Community Service in 1993 and 1994, and during that time also collected data directly from Youth Corps programs for the impact analysis and cost-benefit assessment. In addition, the authors talked with several program staff and participants. The study found that although program completion rates were low, most participants felt the program was helpful and sponsors and beneficiaries were generally satisfied with youths’ performance. The program had a significant impact on the employment and earnings of corps members. The analysis of program costs and benefits showed aggregate benefits to society.","Youth Corps Programs","Mentoring Youth programs","Youth, Disconnected youth, Low-skilled, Low income",,"United States",1997,http://www.abtassociates.com/reports/Youth-Corps.pdf,"Opportunities for Youth Review Protocol" "The effect of OSHA lead exposure in construction standard on blood lead levels among iron workers employed in bridge rehabilitation","Levin, S., Goldberg, M., & Doucette, J. (1997). The effect of the OSHA lead exposure in construction standard on blood lead levels among iron workers employed in bridge rehabilitation. American Journal of Industrial Medicine, 31(3), 303–309.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of the Lead Exposure in Construction Standard imposed by the Occupational Safety & Health Administration (OSHA), which was introduced in May 1993, on the blood lead levels of iron workers employed in the renovation of a large, lead-painted, steel bridge in New York City in 1993 and 1994. The study compared the blood lead levels of iron workers in 1993 and 1994, before and after the introduction of OSHA’s Lead Exposure in Construction Standard. The study found that there was no significant difference in the maximum blood lead levels of iron workers from 1993 to 1994, but there was a decline in the increment in blood lead levels, defined as the difference between the maximum and minimum levels. The quality of causal evidence presented in this study is low. This means we are not confident that any differences in blood lead levels of iron workers before and after the introduction of the OSHA Lead Exposure in Construction Standard were attributable solely to the introduction of the standard.","OSHA Enforcement Activities","Safety Workplace injury",Employed,Construction,"United States",1997,,"OSHA Enforcement Review Protocol" "Falls in Construction: Injury Rates for OSHA-Inspected Employers Before and After Citation for Violating the Washington State Fall Protection Standard","Nelson, N., Kaufman, J., Kalat, J., & Silverstein, B. (1997). Falls in construction: Injury rates for OSHA-inspected employers before and after citation for violating the Washington State Fall Protection Standard. American Journal of Industrial Medicine, 31(3), 296-302.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine the effect of receiving an OSHA citation for violating the Washington State Fall Protection Standard on injury claim rates among construction employers. The study used a regression model to compare injury claim rates for construction employers that had received a citation for violating the Washington State Fall Protection Standard between 1991 and 1992 to rates for those that had not received a citation for violating this standard during this time period. The study found that being cited for violating the falls in construction standard was associated with a statistically significant increase in the likelihood of experiencing a subsequent reduction in injury claim rates compared to construction employers that were not cited. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in the likelihood of experiencing a reduction in injury rates between employers that received a citation for violating the Washington State Fall Protection Standard and employers that did not receive such a citation are attributable to the citations.","OSHA Enforcement Activities","Other health and safety Workers compensation Workplace injury",Employed,Construction,"United States",1997,,"OSHA Enforcement Review Protocol" "Net impact evaluation of retraining under ESHB 1988.","Jacobson, L., & LaLonde, R. (1997). Net impact evaluation of retraining under ESHB 1988. Retrieved from https://files.eric.ed.gov/fulltext/ED413525.pdf","Community College","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages","Summary: The study's objective was to examine the impact of the Washington State Employment and Training Act of 1993 (ESHB 1988) on earnings. The study used a nonexperimental design to compare the earnings of displaced workers who received re-training to those who did not receive re-training. Using administrative data from Washington State's Employment Security Department and transcript data from the community colleges, the authors conducted statistical models to examine the differences between groups. The study found that participation in the re-training program increased long-term earnings; however, the authors did not provide a test of statistical significance. The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design; this is the highest causal evidence rating possible for a nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Washington State Employment and Training Act of 1993 (ESHB 1988), but other factors might also have contributed.","Washington State Employment and Training Act of 1993 (ESHB 1988)","Employment and Training Services Training and Education Basic skills Capacity building programs Community college education and other classroom training","Adult, Dislocated or displaced worker, Low-skilled",,"United States",1997,https://files.eric.ed.gov/fulltext/ED413525.pdf,"Community College Review Protocol" "Evaluation of the Maryland Unemployment Insurance Work Search Demonstration","Benus, J., Johnson, T., Klepinger, D., & Joesch, J. (1997). Evaluation of the Maryland Unemployment Insurance Work Search Demonstration. Prepared for the Maryland Department of Labor, Licensing and Regulation, 1-43. Baltimore, MD: Maryland Department of Labor.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study assessed the effectiveness of four interventions designed to encourage rapid reemployment among Unemployment Insurance (UI) recipients. The study was a randomized controlled trial with more than 27,000 participants. Participants were randomly assigned to one of four treatment groups, each of which had different work search requirements, or to a control group that abided by existing requirements for receipt of UI benefits. The study used state UI administrative records to estimate impacts on employment, earnings, and UI benefit receipt. The authors found that the three treatments that imposed stricter work search requirements than those imposed on the control group reduced the amount of benefits received by $75 to $116 and duration of benefit receipt by 0.6 to 0.9 weeks, on average. There were no statistically significant differences between treatment and control groups on employment or earnings. The quality of causal evidence presented in this report is high because it was a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the work search treatments, and not to other factors.","the Maryland Unemployment Insurance Work Search Demonstration","Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1997,http://wdr.doleta.gov/owsdrr/98-2/98-2.pdf,"Reemployment Review Protocol" "The New Jersey Unemployment Insurance Reemployment Demonstration Project: Six-year follow-up and summary report","Corson, W., & Haimson, J. (1996). The New Jersey Unemployment Insurance Reemployment Demonstration Project: Six-year follow-up and summary report. Revised edition. Unemployment Insurance Occasional Paper 96-2. Washington, DC: U.S. Department of Labor, Employment and Training Administration.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the long-term impacts of the New Jersey Unemployment Insurance Reemployment Demonstration Project on the unemployment insurance (UI) receipt, employment, and earnings of UI claimants in New Jersey approximately six years after program enrollment. In this demonstration, about 11,000 UI claimants were randomized into one of three treatment groups, all of which received some variation of Job Search Assistance (JSA) services, or into the control group, which could receive only existing services in the community. For all participants, the study team collected administrative data, which provided information on UI receipt, employment, and earnings. The study found that the treatment group that received JSA services and a reemployment bonus had a statistically significant reduction in UI dollars received and UI weeks paid, compared with the control group. There were no significant impacts on the probability of working, level of earnings, or weeks worked in the long run for any of the treatment groups. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the interventions studied, and not to other factors.","JSA services","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1996,http://workforcesecurity.doleta.gov/dmstree/op/op95/op_02-95.pdf,"Job Search Assistance, Reemployment Review Protocol" "Outcome evaluation of the Army Career and Alumni Program’s Job Assistance Centers (Study Report 96-04)","Sadacca, R., Laurence, J. H., DiFazio, A. S., Rauch, H. J., & Hintz, D. W. (1995) Outcome evaluation of the Army Career and Alumni Program’s Job Assistance Centers (Study Report 96-04). Alexandria, VA: Human Resources Research Organization.",Veterans,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Mixed impacts Earnings and wages Public benefits receipt-Low-Mixed impacts Public benefit receipt","Summary: The study’s objective was to examine the impact Job Assistance Centers (JACs) and the Transition Assistance Program (TAP) offered to ex-servicemembers had on their earnings and public benefits receipt. The study used a nonexperimental analysis to compare the outcomes of ex-servicemembers transitioning from military to civilian jobs who received various JAC or TAP services to the outcomes of those who did not receive JAC or TAP services. The authors used administrative records from the U.S. Department of Defense, the U.S. Army, and JACs, as well as data from a follow-up survey. The study showed mixed findings on the relationships between receipt of JAC and TAP services and earnings, as well as on the relationships between receipt of JAC and TAP services and receipt of unemployment compensation. The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to JACs and TAP; other factors are likely to have contributed.","Job Assistance Centers and the Transition Assistance Program","Job search assistance and supportive services Other employment and reemployment Veterans' reemployment","Dislocated or displaced worker, Veteran or military",,"United States",1995,,"Employment and Training Review Protocol" "Lessons from the US unemployment insurance experiments","Meyer, B. (1995). Lessons from the US unemployment insurance experiments. Journal of Economic Literature, 33(1), 91-131.","Behavioral Insights","Study Type: Descriptive Analysis",,,"Summary: The article provided an overview of experimental evaluations of unemployment insurance (UI) reforms conducted from 1977 to 1992 in the United States. These reforms typically tried to improve the employment prospects of beneficiaries and reduce UI costs. The author reviewed 10 randomized controlled trials of UI reforms—4 cash bonus experiments and 6 job-search experiments—and provided a comparative analysis on how these reforms affected average weeks of UI benefits paid, UI program costs, and recipients’ earnings, when available. The cash bonus experiments offered a cash payment to participants who quickly found and kept a job for an indicated length of time, although the details varied greatly across experiments. Job-search assistance reforms also varied but typically involved changes in the way job-search programs were implemented in terms of services offered and reporting and participation requirements. The review found that some cash bonus reforms reduced the average number of weeks participants spent on UI by a statistically significant margin, compared with the control group. The review also found that some job-search assistance reforms reduced participants’ average number of weeks on UI and increased average quarterly earnings compared with the control group; however, not all findings for all reforms were statistically significant.",,"Disability insurance Job search assistance and supportive services","Adult, Dislocated or displaced worker, Unemployed",,"United States",1995,,"Behavioral Insights Review Protocol" "Evaluating pooled evidence from the reemployment bonus experiments","Decker, P., & O’Leary, C. (1995). Evaluating pooled evidence from the reemployment bonus experiments. The Journal of Human Resources, 30(3), 534-550.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of Pennsylvania and Washington programs that aimed to encourage faster reemployment among Unemployment Insurance (UI) recipients by offering them financial incentives for faster reemployment. Randomized controlled trials were conducted separately in both Pennsylvania and Washington State and included a total of about 27,500 UI claimants. The authors estimated impacts of the reemployment bonus programs on UI benefits receipt and earnings using state UI administrative records. The study found that, on average, UI claimants who had been randomly assigned to receive any of the reemployment bonus offers received 0.5 fewer weeks and $85 less in total UI benefits than those assigned to the control group. However, they were no more or less likely to exhaust UI benefits or have higher earnings in the year following random assignment than control group members. The quality of causal evidence presented in this report is high because it is based on pooled analysis of two well-implemented randomized controlled trials. This means we are confident that the estimated effects are attributable to reemployment bonus incentives, and not to other factors.","the Pennsylvania Reemployment Bonus Demonstration and the Washington Reemployment Bonus Experiment","Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1995,,"Reemployment Review Protocol" "The Jobs I Preventive Intervention for Unemployed Individuals: Short- and long-term effects on reemployment and mental health","Vinokur, A., Price, R., Caplan, R., van Ryn, M.,& Curran, J. (1995). The Jobs I Preventive Intervention for Unemployed Individuals: Short- and long-term effects on reemployment and mental health. In L.R. Murphy, J.J. Hurrell, Jr., S.L. Sauter, & G.P. Keita (Eds.), Job Stress Interventions. Washington, DC: American Psychological Association (125-138).",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-Favorable impacts Earnings and wages","Summary: The study’s objective was to examine the impact of the Jobs Program on short-term earnings. The authors conducted a randomized controlled trial (RCT). Researchers administered a pre-test two weeks before the intervention and two post-tests one and four months after the intervention, comparing the outcomes of Jobs Program participants against those of a control group. The study found statistically significant associations between being offered access to the Jobs Program and increased earnings one and four months later. The quality of causal evidence presented in this report is low because the study was an RCT with post-intervention subsampling based on participation in the intervention. In addition, the authors did not include sufficient controls in their analysis. This means we are not confident that any estimated effects would be attributable to the Jobs Program; other factors are likely to have contributed.","the Jobs Program","Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",1995,,"Reemployment Review Protocol" "Expressive writing and coping with job loss","Spera, S., Buhrfeind, E., & Pennebaker, J. (1994). Expressive writing and coping with job loss. Academy of Management Journal, 37(3), 722-733.","Behavioral Insights","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment","Summary: The study’s objective was to examine the impact of expressive writing activities on the reemployment of people who had recently lost their jobs. The authors randomly assigned 41 people who had recently been laid off to either a treatment group, which was instructed to write daily about their feelings on their job loss, or to a control group, which was instructed to write about their plans for the day, but not about their feelings. Another 22 participants were assigned (non-randomly) to a comparison group that did not participate in a writing activity. The authors compared outcomes for the three study groups collected from administrative records and four surveys. The study found that participants who wrote about their feelings and attitudes about being laid off were more likely to find employment eight months after the program than those who wrote about other topics and those who did no writing activities. The quality of causal evidence presented in this report is high for comparisons between the treatment group and the writing control group because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to expressive writing, and not to other factors. However, the quality of causal evidence is moderate for comparisons between the treatment group and the non-writing comparison group because there was non-random assignment to this group. This means we are somewhat confident that the estimated effects are attributable to expressive writing, but other factors might also have contributed.",,"Behavioral Interventions","Unemployed, Adult",,"United States",1994,,"Behavioral Insights Review Protocol" "Self-employment as a reemployment option: Demonstration results and national legislation","Benus, J., Johnson, T., Wood, M, & Grover, N. (1994). Self-employment as a reemployment option: Demonstration results and national legislation. Unemployment Insurance Occasional Paper, 94(3), 1-61.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study estimated the impacts of Washington State’s Self-Employment and Enterprise Demonstration (SEED) and the Massachusetts Enterprise Project. Both programs sought to determine how feasible self-employment was as a reemployment option for Unemployment Insurance (UI) recipients. This profile focused only on the program design and outcomes of the Enterprise Project. The Enterprise Project was evaluated using a randomized controlled trial in which program applicants were randomly assigned to either a treatment group, which could receive Enterprise Project services, or a control group, which received UI benefits as usual. Outcomes of interest included measures of employment, self-employment, earnings, and UI benefit receipt. The study found that applicants in the treatment group entered self-employment and wage employment at higher rates than those in the control group. They also had higher earnings, on average, and reduced UI benefit duration and amount received. The quality of the causal evidence presented in this study is high. This means we are confident that the estimated effects are attributable to the Enterprise Project, and not to other factors.","the Enterprise Project Demonstration","Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1994,http://workforcesecurity.doleta.gov/dmstree/op/op94/op_03-94.pdf,"Reemployment Review Protocol" "The impact of Reemployment Bonuses on Insured Unemployment in the New Jersey and Illinois Reemployment Bonus Experiments","Decker, P. (1994). The impact of Reemployment Bonuses on Insured Unemployment in the New Jersey and Illinois Reemployment Bonus Experiments. Journal of Human Resources, 29(3), 718-741.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: This study’s objective was to assess the effectiveness of New Jersey and Illinois programs that aimed to encourage faster reemployment among Unemployment Insurance (UI) recipients by offering them financial incentives for faster reemployment. Each state conducted separate randomized controlled trials. The authors calculated weekly UI exit rates using state UI administrative records. The study found that both programs significantly increased the weekly rates at which bonus-eligible UI claimants left the UI rolls during the qualification period relative to claimants who were not bonus-eligible, by 14 percent in New Jersey and 18 percent in Illinois. The quality of causal evidence presented in this report is high because it is based on two well-implemented randomized controlled trials. This means we are confident that the estimated effects are attributable to reemployment bonus incentives and not to other factors.","the New Jersey Unemployment Insurance Reemployment Demonstration Project and Illinois Job Search Incentive Experiment","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1994,,"Reemployment Review Protocol" "First impact analysis of the Washington State Self-Employment and Enterprise Development (SEED) demonstration","Benus, J., Johnson, T., & Wood, M. (1994). First impact analysis of the Washington State Self-Employment and Enterprise Development (SEED) demonstration. Unemployment Insurance Occasional Paper, 94(1), 1-172.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Mixed impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study examined the impact of Washington State’s Self-Employment and Enterprise Demonstration (SEED) on reemployment via self-employment for Unemployment Insurance (UI) claimants. SEED was evaluated using a randomized controlled trial in which SEED applicants were randomly assigned to a treatment group, which could receive several types of business start-up training and support services, or a control group, which received UI benefits as usual. Outcomes of interest included measures of employment, self-employment, earnings, and UI benefit receipt. The study found that applicants in the treatment group entered self-employment at significantly higher and faster rates than those in the control group did. SEED participants also earned significantly more from self-employment than nonparticipants did. The quality of the causal evidence presented in this study is high because it was a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to SEED, and not other factors.","Washington State’s Self-Employment and Enterprise Demonstration","Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1994,http://wdr.doleta.gov/research/FullText_Documents/op_01-94.pdf,"Reemployment Review Protocol" "Does regulatory enforcement work? A panel analysis of OSHA enforcement","Gray, W., & Scholz, J. (1993). Does regulatory enforcement work? A panel analysis of OSHA enforcement. Law & Society Review, 27(1), 177-213.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to determine the effect on injury rates of OSHA inspections that resulted in penalties during the 1980s. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The study used a regression model to compare the percentage change in injury rates when firms had received inspections that resulted in penalties during the past four years and when the same firms had not received penalties during the past four years. The study found that inspections that resulted in penalties were associated with statistically significant reductions in lost workday injuries (by 22 percent) and lost workdays (by 20 percent) during the three years following the inspection. The quality of the causal evidence presented in this study is low. This means we are not confident that the changes in injury rates within firms when they received inspections that resulted in penalties are attributable solely to these penalties.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,,"United States",1993,,"OSHA Enforcement Review Protocol" "The Washington Reemployment Bonus Experiment: Final report","Spiegelman, R., O’Leary, C., & Kline, K. (1992). The Washington Reemployment Bonus Experiment: Final report. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to assess the impacts of the Washington Reemployment Bonus experiment, which aimed to encourage faster reemployment rates among Unemployment Insurance (UI) claimants by offering them financial incentives, on the UI benefits receipt of UI claimants. In this demonstration about 15,500 eligible claimants were randomized into one of six treatment groups or a control group. The six treatment groups were offered different combinations of reemployment bonus amounts and periods within which they were required to find a job in order to claim the bonus. For all participants, the study team collected administrative data, which provided information on UI benefits receipt. The study found that three of the bonus treatments reduced the weeks of UI benefits and total UI benefits dollars received, but the other three did not. Pooling across all bonus treatments, treatment group members received 0.4 fewer weeks and $65 less in UI benefits, on average, than their control group counterparts. The quality of causal evidence presented in this study is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the reemployment bonuses, and not to other factors.","the Washington Reemployment Bonus Experiment","Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1992,http://research.upjohn.org/externalpapers/44/,"Reemployment Review Protocol" "Time-varying effects of recall expectation, a reemployment bonus, and job counseling on unemployment durations","Anderson, P. (1992). Time-varying effects of recall expectation, a reemployment bonus, and job counseling on unemployment durations. Journal of Labor Economics, 10 (1), 99-115.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study assessed the effectiveness of a reemployment bonus on the reemployment rates of Unemployment Insurance (UI) claimants with differing expectations of being recalled to their previous jobs. The study used data from the New Jersey Unemployment Insurance Reemployment Demonstration Project. For the demonstration, about 11,000 UI claimants were randomized into one of three treatment groups, all of which received some variation of Job Search Assistance (JSA) services, or into the control group, which could receive only existing services in the community. The author estimated weekly UI exit rates from state UI administrative records. The study found that the JSA-plus-reemployment-bonus group members were more likely than members of the JSA-only group to leave UI for a new job during weeks 7 through 12, which corresponded to the first half of the bonus qualification period, suggesting the bonuses were effective in spurring UI claimants to find new jobs more quickly. The quality of causal evidence presented in this report is moderate because it is a secondary analysis of data from a well-implemented randomized controlled trial for which attrition cannot be calculated. However, the author included sufficient statistical controls to receive a moderate evidence rating. This means we have some confidence that the estimated effects are attributable to the interventions studied, although other factors also could have contributed.","the New Jersey Unemployment Insurance Reemployment Demonstration Project","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1992,,"Job Search Assistance, Reemployment Review Protocol" "Pennsylvania Reemployment Bonus Demonstration final report","Corson, W., Decker, P., Dunstan, S., & Kerachsky, S. (1992). Pennsylvania Reemployment Bonus Demonstration final report. Unemployment Insurance Occasional Paper 92-1. Washington, DC: Employment and Training Administration, U.S. Department of Labor.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Mixed impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impacts of the Pennsylvania Reemployment Bonus Demonstration Project on the unemployment insurance (UI) receipt, employment, and earnings of UI claimants approximately one year after program enrollment. In this demonstration, about 15,000 eligible claimants were randomly assigned to one of six treatment groups with varying financial incentives and qualification periods or a control group. For all participants, the study team collected administrative data, which provided information on UI benefits receipt, employment, and earnings. The study team also surveyed a subsample of approximately 5,000 participants about their experiences in the year following random assignment. The study found that two of the bonus treatments reduced the weeks of UI benefits and total UI benefit dollars received, but the others did not. The study found very few impacts of the bonus treatments on claimants’ reemployment rates or earnings. The quality of causal evidence is high for UI benefits receipt outcomes because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to reemployment bonus incentives, and not to other factors. However, the quality of causal evidence is moderate for the employment and earnings outcomes because study attrition could not be determined for those outcomes.","the Pennsylvania Reemployment Bonus Demonstration","Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1992,http://www.mathematica-mpr.com/~/media/publications/PDFs/pennreemploy.pdf,"Reemployment Review Protocol" "Longitudinal patterns of compliance with Occupational Safety and Health Administration health and safety regulations in the manufacturing sector","Gray, W., & Jones, C. (1991). Longitudinal patterns of compliance with Occupational Safety and Health Administration health and safety regulations in the manufacturing sector. Journal of Human Resources, 26(4), 623-653.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Compliance-Mod/high-Favorable impacts Compliance","Summary: The study’s objective was to examine the effect of past Occupational Safety & Health Administration (OSHA) inspections on the number of violations found in subsequent inspections. The study used a regression model to compare the number of hazards found at the first, second, and higher-order inspections conducted by OSHA from 1972 to 1983. Although OSHA no longer operates as it did during the period of this study, the study provides interesting historical context. The study found that the number of workplace hazards cited decreased with each additional inspection. The largest drop occurred from the first to the second inspections. The quality of causal evidence presented in this study is moderate because the study used a well-implemented nonexperimental design. This means we have confidence that the effects estimated in this study are attributable at least in part to OSHA inspections. However, as with any nonexperimental design, other factors not accounted for in the study could also have contributed to the estimated effects.","OSHA Enforcement Activities","Inspections Safety",Employed,,"United States",1991,,"OSHA Enforcement Review Protocol" "Re-estimating OSHA’s Effects: Have the data changed?","Ruser, J. & Smith, R. (1991). Re-estimating OSHA’s effects: Have the data changed? Journal of Human Resources, 26(2), 212-235.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-No impacts Health and safety","Summary: The study’s objective was to determine the effect of OSHA inspections and inspection procedures on injury rates in manufacturing firms between 1980 and 1985. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. For their main analysis, the authors used a regression model to compare differences in injury rates for manufacturing plants that received inspections in March and April (“early”) and those that received inspections in November and December (“late”) of the same year. The study found that, relative to late OSHA inspections, early OSHA inspections were not associated with a statistically significantly reduction in injury rates within the year of inspection. The quality of the causal evidence presented in this study is low. This means we are not confident that the estimated relationships in this study are the result of OSHA inspections and procedures.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,Manufacturing,"United States",1991,,"OSHA Enforcement Review Protocol" "Are OSHA Health Inspections Effective? A Longitudinal Study in the Manufacturing Sector ","Gray, W., & Jones, C. (1991). Are OSHA health inspections effective? A longitudinal study in the manufacturing sector. Review of Economics & Statistics, 73(3), 504.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Compliance-Mod/high-Favorable impacts Compliance","Summary: The study’s objective was to examine the effect of past Occupational Safety & Health Administration (OSHA) health inspections on the number of violations found in subsequent inspections. The study used regression models to compare the number of hazards found at the first, second, and higher-order inspections conducted by OSHA from 1972 to 1983. Although OSHA no longer operates as it did during the period of this study, the study provides interesting historical context. The study found that the number of workplace hazards cited decreased with each additional OSHA inspection. The largest drop occurred between the first and second inspections. The quality of causal evidence presented in this study is moderate because the study used a well-conducted nonexperimental design. This means we have confidence that the estimated effects are attributable at least in part to OSHA inspections. However, as with any nonexperimental design, other factors not accounted for in the study could also have contributed to the estimated effects.","OSHA Enforcement Activities","Inspections Safety",Employed,,"United States",1991,,"OSHA Enforcement Review Protocol" "Long-term follow-up and benefit-cost analysis of the Jobs Program","Vinokur, A., van Ryn, M., Gramlich, E., & Price, R. (1991). Long-term follow-up and benefit-cost analysis of the Jobs Program. Journal of Applied Psychology, 76(2), 213-219.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Earnings and wages-Low-No impacts Earnings and wages Employment-Low-No impacts Employment","Summary: The study’s objective was to examine the impact of the Jobs Program on long-term earnings and monthly hours worked. The authors conducted a randomized controlled trial (RCT). Researchers administered a pre-test two weeks before the intervention and three post-tests at 1, 4, and 30 months after the intervention comparing the outcomes of Jobs Program participants against a control group. The study found no statistically significant relationships between the Jobs Program and increased long-term earnings or time in work during the follow-up period. The quality of causal evidence presented in this report is low because the study was an RCT with post-intervention subsampling based on participation in the intervention. In addition, the authors did not include sufficient controls in their analysis. This means we are not confident that any estimated effects would be attributable to the Jobs Program; however, the study found no statistically significant effects.","the Jobs Program","Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",1991,,"Reemployment Review Protocol" "Evaluation of the impacts of the Washington Alternative Work Search Experiment","Johnson, T., & Klepinger, D. (1991). Evaluation of the impacts of the Washington Alternative Work Search Experiment. Unemployment Insurance Occasional Paper, 91(4), 1–90.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study assessed the impacts of four different work search policy approaches implemented through the Washington Alternative Work Search Experiment on Unemployment Insurance (UI) benefits receipt, employment, and earnings outcomes. Eligible UI claimants were randomly assigned to one of three treatment groups or a control group. The four study groups faced different work search requirements to continue receiving UI benefits. For all participants, the study team collected administrative data, which provided information on UI benefits receipt, employment, and earnings. The study found that UI claimants facing less-stringent work search requirements received more UI benefits for longer and were more likely to exhaust their benefits, on average, than those facing standard work search requirements. There were few differences in UI benefits, employment, or earnings among claimants subject to more stringent work search requirements compared with those facing standard work search requirements. The quality of causal evidence presented in this study is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the alternative work search requirements, and not to other factors.","the Washington Alternative Work Search Experiment","Job search assistance and supportive services Unemployment Insurance","Dislocated or displaced worker, Unemployed",,"United States",1991,,"Reemployment Review Protocol" "The New Jersey Unemployment Insurance Reemployment Demonstration Project: Follow-up report","Anderson, P., Corson, W., & Decker, P. (1991). The New Jersey Unemployment Insurance Reemployment Demonstration Project: Follow-up report. Unemployment Insurance Occasional Paper 91-1. Washington, DC: U.S. Department of Labor, Employment and Training Administration.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the New Jersey Unemployment Insurance Reemployment Demonstration Project on the unemployment insurance (UI) receipt, employment, and earnings of UI claimants in New Jersey three to four years after program enrollment. In this demonstration, about 11,000 UI claimants were randomized into one of three treatment groups, all of which received some variation of Job Search Assistance (JSA) services, or into the control group, which could receive only existing services in the community. For all participants, the study team collected administrative data, which provided information on UI receipt, employment, and earnings. The study found that, over the claim year and three follow-up years, the JSA-plus-reemployment-bonus group had statistically significant differences from the control group on UI dollars received ($293 fewer than control) and UI weeks paid (1.6 fewer weeks than control). The study found few statistically significant impacts on the probability of working, earnings, or weeks worked for any of the treatment groups compared with the control group. The quality of causal evidence presented in this report is high because it based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the interventions studied, and not to other factors.","JSA services","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1991,http://workforcesecurity.doleta.gov/dmstree/op/op91/op_01-91.pdf,"Job Search Assistance, Reemployment Review Protocol" "The economic impact of the Nevada Claimant Employment Program","Hanna, J., & Turney, Z. (1990). The economic impact of the Nevada Claimant Employment Program. Unemployment Insurance Occasional Paper 90(4), 79-92.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study determined the effect of reemployment services, particularly job training for those with insufficient skills or education, on Unemployment Insurance (UI) receipt duration. The study was a randomized controlled trial (RCT), with sample members’ outcomes drawn from Nevada state UI records. The study found that reemployment services reduced UI receipt duration by 1.6 weeks, on average, during the first year. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the Nevada Claimant Employment Project; other factors are likely to have contributed.","JSA services","RESEA Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",1990,,"Reemployment Review Protocol" "Back to work: Testing reemployment services for displaced workers","Bloom, H. (1990). Back to work: Testing reemployment services for displaced workers. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-No impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Texas Worker Adjustment Demonstration on the Unemployment Insurance (UI) receipt, employment, and earnings of UI claimants one year after program enrollment. In this demonstration, about 2,200 UI claimants across three sites were randomized into a treatment group, which received job-search assistance services and, if needed, occupational training, or into the control group, which could receive only existing services in the community. For all participants, the study team collected administrative data and conducted a one-year follow up survey. The study found that women who received services at the two El Paso sites earned $987 more and received $193 less in UI benefits in the first year, and were 10 to 20 percentage points more likely to be employed in the first three quarters after entering the program, compared with the control group. The study also found that men in the treatment groups at all three sites received $143 less in UI benefits than men in the control group, but there were no other statistically significant impacts one year after random assignment. The quality of causal evidence presented in this report is moderate because it is an analysis of data from a randomized controlled trial for which attrition cannot be calculated, but for which the author included sufficient statistical controls. This means we have some confidence that the estimated effects are attributable to the demonstration services, although other factors also could have contributed.","JSA services","Job search assistance and supportive services Basic skills Community college education and other classroom training Work based and other occupational training","Dislocated or displaced worker, Unemployed",,"United States",1990,,"Job Search Assistance, Reemployment Review Protocol" "The New Jersey Unemployment Insurance Reemployment Demonstration Project: Final evaluation report","Corson, W., Decker, P.T., Dunstan, S.M., Gordon, A.R., Anderson, P., & Homrighausen, J. (1989). The New Jersey Unemployment Insurance Reemployment Demonstration Project: Final evaluation report. Princeton, NJ: Mathematica Policy Research.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-Favorable impacts Earnings and wages Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the short-term impacts of the New Jersey Unemployment Insurance Reemployment Demonstration Project on the unemployment insurance (UI) receipt, employment, and earnings of UI claimants in New Jersey approximately one year after program enrollment. In this demonstration, about 11,000 UI claimants were randomized into one of three treatment groups, all of which received some variation of Job Search Assistance (JSA) services, or into the control group, which could receive only existing services in the community. For all participants, the study team collected administrative data, which provided information on UI receipt, employment, and earnings. The study team also surveyed a subsample of roughly 7,500 participants about their experiences in the year following random assignment. The study found that all three treatments reduced at least one measure of UI benefits received in the benefit year. According to the survey data, the JSA-only and JSA-plus-bonus groups had higher employment and earnings in the year following participants’ initial UI claims than control group members (no difference was found for the JSA-plus-training group). However, these impacts were not observed in the administrative data. The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the interventions studied, and not to other factors.","JSA services","Job search assistance and supportive services Other employment and reemployment Other training and education","Dislocated or displaced worker, Unemployed",,"United States",1989,http://workforcesecurity.doleta.gov/dmstree/op/op89/op_03-89.pdf,"Job Search Assistance, Reemployment Review Protocol" "Requiring Unemployment Insurance recipients to register with the Public Employment Service","Director, S., & Englander, F. (1988). Requiring Unemployment Insurance recipients to register with the Public Employment Service. The Journal of Risk and Insurance, 55(2), 245-258.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of New Jersey’s repeal of mandatory Employment Services (ES) registration for Unemployment Insurance (UI) claimants on the length of claimants’ UI spells and their probability of exhausting available UI benefits. The authors compared UI outcomes before and after repeal using an interrupted time series design based on monthly, state-level time series data from 1971 to 1981. The study found that the UI exhaustion rate and weeks of UI benefits received were lower when ES registration was mandatory than when it was voluntary. The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to mandatory ES registration; other factors are likely to have contributed.","More stringent work search requirements","RESEA Unemployment Insurance Wagner-Peyser Act","Dislocated or displaced worker, Unemployed",,"United States",1988,,"Reemployment Review Protocol" "The effect of OSHA records-check inspections on reported occupational injuries in manufacturing establishments","Ruser, J., & Smith, R. (1988). The effect of OSHA records-check inspections on reported occupational injuries in manufacturing establishments. Journal of Risk and Uncertainty, 1(4), 415–435.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Mixed impacts Health and safety","Summary: The study’s objective was to determine whether OSHA’s records-check procedure—which was introduced in 1981 in states with federal OSHA enforcement and targeted inspections based on reports of past injuries—caused manufacturing plants to underreport injuries. Although OSHA no longer operates as it did around the time of the records-check procedure’s implementation, this study provides historical context for changes that were later made to the program. The authors used several approaches, the strongest of which compared the percentage change in lost-workday injury rates in the years when the plant’s state had a records-check procedure and the years when the state did not have this procedure. The study found no statistically significant differences between the percentage change in reported injury rates in plants in states with the records-check procedure and plants in states without this procedure. However, plants in risky industries in records-check states reported a statistically significant 4.7 percent reduction in injury rates. The quality of the causal evidence presented in this study is low. This means we are not confident that any changes in reported injuries within plants in records-check states or in risky industries in records-check states could be attributed to the records-check procedure.","OSHA Enforcement Activities","Other health and safety Workplace injury",Employed,Manufacturing,"United States",1988,,"OSHA Enforcement Review Protocol" "The Illinois Unemployment Insurance Incentive Experiments","Spiegelman, R., & Woodbury, S. (1987). The Illinois Unemployment Insurance Incentive Experiments. Kalamazoo, MI: W.E. UpJohn Institute for Employment Research.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Earnings and wages-Mod/high-No impacts Earnings and wages Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to examine the impact of the Job Search Incentive Experiment and the Hiring Incentive Experiment in Illinois on Unemployment Insurance (UI) benefit receipt and earnings. In this study, about 17,000 new UI claimants were randomly assigned to one of two treatment groups, both of which could participate in a reemployment bonus program, or the control group, which faced existing UI program rules and could not receive a reemployment bonus. For all participants, the study team collected administrative data on UI benefits receipt. The study found that those in the job search incentive group had a significant reduction in UI benefits received, length of UI receipt, and proportion of claimants exhausting UI benefits, compared with the control group. Those in the hiring incentive group had a significant reduction in UI benefits received relative to the control group, but no other outcomes.  The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the reemployment bonuses, and not to other factors.","the Illinois Job Search Incentive and Hiring Incentive Experiments","Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1987,,"Reemployment Review Protocol" "Evaluation of the perceivable demand list pilot project","Behrens, J. (1987). Evaluation of the perceivable demand list pilot project. Trenton, NJ: New Jersey Department of Labor.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The report’s objective was to evaluate the Perceivable Demand List (PDL) Pilot Project on the duration of unemployment insurance (UI) receipt among recent beneficiaries laid off from high-demand occupations.  For this evaluation, UI recipients in Hackensack, New Jersey, who were recently laid off from high-demand occupations, as determined by the New Jersey Department of Labor, were randomly assigned to either participate in the PDL pilot or receive services as usual. The PDL pilot reduced UI receipt by an average of 2.6 weeks, reduced total UI benefits received by $340, and reduced the proportion of UI recipients who exhausted their benefits by 9.8 percentage points. The quality of the causal evidence presented in this report is high because it was a well-implemented randomized controlled trial. This means we are confident that the estimated impacts are attributable solely to the PDL Pilot, not other factors.","More stringent work search requirements","RESEA Other employment and reemployment Wagner-Peyser Act","Dislocated or displaced worker, Unemployed",,"United States",1987,,"Reemployment Review Protocol" "Safety Regulation and Workplace Injuries","Curington, W. (1986). Safety regulation and workplace injuries. Southern Economic Journal, 53(1), 51.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of the introduction of OSHA regulation in 1971 on the severity and frequency of workplace injuries in manufacturing industries in New York. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The study used a regression model to examine the change in the frequency and severity of injuries from the period 1964 to 1970 compared with 1971 to 1976, which represent the time periods before and after the implementation of OSHA standards. The study found that the implementation of OSHA standards was not associated with a statistically significant reduction in the average frequency or the severity of all injuries. The study found a statistically significant association between implementation of OSHA standards and a reduction in ""struck by machine"" injuries. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in injuries from the period 1964 to 1970 compared with 1971 to 1976 are attributable to the introduction of OSHA standards.","OSHA Enforcement Activities","Other health and safety Workplace injury",Employed,Manufacturing,"United States",1986,,"OSHA Enforcement Review Protocol" "The impact of Occupational Safety and Health regulations, 1973-1983","Viscusi, W. (1986). The impact of Occupational Safety and Health regulations, 1973-1983. RAND Journal of Economics, 17(4), 567–580.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-Mixed impacts Health and safety","Summary: The study’s objective was to examine changes in the effects of OSHA inspections and penalties on injury rates in manufacturing industries between 1973 and 1983. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The study used a regression model to examine the changes in injury rates in manufacturing industries associated with higher and lower rates of OSHA inspections and penalties during the prior two years. The study found that the evidence of the effectiveness of OSHA inspections and penalties in reducing injuries was mixed, and the effects of OSHA enforcement activities on most outcomes varied over time. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in injuries within industries are attributable to the inspections or penalties.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,Manufacturing,"United States",1986,,"OSHA Enforcement Review Protocol" "Evaluation of the Charleston Claimant Placement and Work Test Demonstration","Corson, W., Long, D., & Nicholson, W. (1985). Evaluation of the Charleston Claimant Placement and Work Test Demonstration. Department of Labor Unemployment Insurance Occasional Paper, 85(2), 1-113.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Employment-Mod/high-Favorable impacts Employment Public benefits receipt-Mod/high-Favorable impacts Public benefit receipt","Summary: The study’s objective was to determine the impact of a more-stringent work test and enhanced employment services on Unemployment Insurance (UI) claimants’ benefits receipt and reemployment outcomes. The authors randomly assigned UI claimants to four conditions, comprising three distinct treatment groups and one control group. Administrative data sources, including UI and Employment Service (ES) records as well as records of claimants’ interactions with program staff provided outcomes for sample members in all four conditions. The study found that UI claimants in the more-intensive treatment conditions—treatment groups 1 and 2—collected UI benefits for fewer weeks than those in the control condition, by 0.76 weeks in treatment group 1 and 0.61 weeks in treatment group 2; these differences were statistically significant. The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Charleston Claimant Placement and Work Test Demonstration, and not to other factors.","JSA services","Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",1985,,"Job Search Assistance, Reemployment Review Protocol" "Direct and indirect effects of regulation: A new look at OSHA’s impact","Bartel, A., & Thomas, L. (1985). Direct and indirect effects of regulation: A new look at OSHA’s impact. Journal of Law and Economics, 28(1), 1–25.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Compliance-Low-Favorable impacts Compliance Health and safety-Low-No impacts Health and safety","Summary: The study’s objective was to determine the effect of Occupational Safety & Health Administration (OSHA) inspections on noncompliance with OSHA standards and the effect of noncompliance on lost-workday injuries in the 1970s. Although OSHA no longer operates as it did during that period, this study provides historical context for later changes made to the program. The study used a structural model to examine differences in the injury rates and noncompliance for industries that received higher rates of OSHA inspections. The study did not find a statistically significant relationship between noncompliance with OSHA standards and workplace injuries. The quality of the causal evidence presented in this study is low. This means we are not confident that the differences in injury rates among industries with higher rates of noncompliance are attributable solely to noncompliance or that differences in noncompliance rates among industries with higher rates of inspections are attributable to inspections.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,,"United States",1985,,"OSHA Enforcement Review Protocol" "Wisconsin Job Service: ERP Pilot Project final report","Wisconsin Department of Industry, Labor, and Human Relations (1984). Wisconsin Job Service: ERP Pilot Project final report. Madison, WI: DILHR.","Job Search Assistance, Reemployment","Study Type: Causal Impact Analysis","Causal Evidence Rating: High Causal Evidence","Public benefits receipt-Mod/high-No impacts Public benefit receipt","Summary: The study estimated the effect of mandatory participation in a job search workshop for Unemployment Insurance (UI) recipients separated indefinitely from their previous employers. The study was a randomized controlled trial (RCT) conducted in six sites in Wisconsin. Eligible UI recipients were randomly assigned to either the treatment group, which was required to attend a six-hour workshop, or a control group, which was not required to attend the workshop. In general, there were no statistically significant differences in duration of UI benefit receipt between treatment and control group members across all six sites or for individual sites. The quality of causal evidence presented in this report is high because it was based on a well-implemented RCT. This means we are confident that the estimated effects are attributable to the job search workshop and not to other factors.","JSA services","Job search assistance and supportive services","Dislocated or displaced worker, Unemployed",,"United States",1984,,"Job Search Assistance, Reemployment Review Protocol" "An assessment of OSHA’s recent effects on injury rates","McCaffrey, D. (1983). An assessment of OSHA’s recent effects on injury rates. Journal of Human Resources, 18(1), 131-146.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Health and safety-Mod/high-No impacts Health and safety","Summary: The study’s objective was to examine the effect of OSHA inspections on injury rates in manufacturing and construction firms from 1976 to 1978. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The author used a regression model to compare the differences in injury rates of manufacturing and construction firms that received inspections in March and April (“early”) with those that received inspections in November and December (“late”). The study found that, relative to late OSHA inspections, early OSHA inspections were not associated with a statistically significantly reduction in injury rates within the year of inspection. The quality of causal evidence for some analyses presented in this study is moderate because the analyses used a well-conducted nonexperimental design. This means we have confidence that these findings provide some evidence that OSHA inspections had no effect on injury rates.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,"Construction, Manufacturing","United States",1983,,"OSHA Enforcement Review Protocol" "Worker injuries: The effects of Workers’ Compensation and OSHA inspections","Robertson, L., & Keeve, J. (1983). Worker injuries: The effects of Workers' Compensation and OSHA inspections. Journal of Health Politics, Policy and Law, 8(3), 581-597.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employment-Low-Favorable impacts Employment Health and safety-Low-Favorable impacts Health and safety","Summary: The study had two objectives: (1) to examine the effect of OSHA citations on injuries (method 1) and (2) to examine the effect of changes in OSHA inspections on changes in days of work lost because of injuries (method 2). Although OSHA no longer operates as it did during the study period, the study provides historical context for the program. Method 1 used a regression model to compare injuries across three firms from 1973 to 1980. Method 2 used a different regression model to compare changes in work days lost because of injuries from 1975 to 1976 to changes in OSHA inspections during the same period, using data at the industry-by-state level. The study found that firms had 35.8 fewer injuries in the years in which they received a citation; however, citations in past years were not significantly related to current year injuries. One additional OHSA inspection was associated with a 2.8 day reduction in the number of work days lost because of injuries.  The quality of causal evidence presented by both methods in this study is low. This means we are not confident that the differences in workplace injuries or lost work days are attributable to OSHA citations or inspections.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,Manufacturing,"United States",1983,,"OSHA Enforcement Review Protocol" "The Impact of OSHA Inspections on Manufacturing Injury Rates","Smith, R (1979). The Impact of OSHA Inspections on Manufacturing Injury Rates. Journal of Human Resources, 14(2),  145-170.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Moderate Causal Evidence","Health and safety-Mod/high-Favorable impacts Health and safety","Summary: The study’s objective was to examine the effect of OSHA inspections on injury rates in manufacturing firms in 1973 and 1974. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The authors used a regression model to compare the differences in injury rates of firms that received inspections in March and April (“early”) and those that received inspections in November and December (“late”). The study found that in 1973, firms receiving early inspections had significantly lower injury rates that year than those that were inspected later in the year. However, in 1974, firms inspected early and late in the year had similar injury rates. The quality of causal evidence presented in this study is moderate because the study used a well-conducted nonexperimental design. This means we have confidence that the effects estimated in this study are attributable at least in part to OSHA inspections. However, as with any nonexperimental study design, other factors not accounted for in the study could also have contributed to the estimated effects.","OSHA Enforcement Activities","Inspections Workplace injury",Employed,Manufacturing,"United States",1979,,"OSHA Enforcement Review Protocol" "The impact of Occupational Safety and Health regulation","Viscusi, W. (1979). The impact of Occupational Safety and Health regulation. The Bell Journal of Economics, 10(1), 117–140.","OSHA Enforcement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Health and safety-Low-No impacts Health and safety","Summary: The study’s objective was to examine the effects of OSHA inspections and penalties at the industry level on workplace injuries between 1972 and 1975. Although OSHA no longer operates as it did during this period, this study provides historical context for changes that were later made to the program. The study used a regression model to examine differences in injury rates for industries and years with higher rates of OSHA inspections and penalties, relative to industries and years with lower rates of OSHA inspections or penalties. The study found that OSHA inspections and proposed penalties were not statistically significantly related to injuries. The quality of causal evidence presented in this study is low. This means we are not confident that the study’s findings provide causal evidence of the effect of inspections on injuries.","OSHA Enforcement Activities","Inspections Other health and safety Workplace injury",Employed,,"United States",1979,http://law.vanderbilt.edu/files/archive/009_Impact_of_Occupational_Safety.pdf,"OSHA Enforcement Review Protocol" "The Nevada claimant placement project.","Steinman, J. (1978). The Nevada claimant placement project. Washington, DC: U.S. Department of Labor, Employment & Training Administration.",Reemployment,"Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Public benefits receipt-Low-Favorable impacts Public benefit receipt","Summary: The study’s objective was to estimate the impact of the Nevada Claimant Placement Project (NCPP), an intervention designed to accelerate labor force reattachment among Unemployment Insurance (UI) claimants through mandatory, intensive case management. The study was a randomized controlled trial; UI claimants were randomly assigned to either a treatment group, which received enhanced case management services, or to a control group, which received typical services available to UI claimants. The author used UI administrative records to compare the UI benefit receipt of treatment group members with that of control group members. The study found that the treatment group members received $318 less in UI benefits than control group members. The quality of the causal evidence presented in this report is low because attrition could not be determined and the study did not include controls for claimant characteristics in its analysis. This means we are not confident that the estimated effects are attributable to the NCPP.","Reemployment Eligibility Assessment","RESEA Job search assistance and supportive services Other employment and reemployment","Dislocated or displaced worker, Unemployed",,"United States",1978,,"Reemployment Review Protocol" "Investor behavior and the purchase of company stock in 401(k) plans—the importance of plan design","Liang, N., & Weisbenner, S. (2002). Investor behavior and the purchase of company stock in 401(k) plans—the importance of plan design. National Bureau of Economic Research working paper 9131. Cambridge, MA: NBER.","Behavioral Finance: Retirement","Study Type: Causal Impact Analysis","Causal Evidence Rating: Low Causal Evidence","Employer benefits receipt-Low-Favorable impacts Employer benefits receipt","Summary: The study’s objective was to determine whether 401(k) plan characteristics, including the number of total investment options offered by a 401(k) plan and the requirements for whether employers’ matching contributions had to be allocated to employer stock, influenced the percentage of employees’ 401(k) plan contributions allocated to employer stock. The authors analyzed differences in portfolio allocations based on the number of funds available and other key plan characteristics using data from Compustat and Securities and Exchange Commission (SEC) forms and regression analyses, controlling for firm-fixed effects. The number of investment options and the percentage of assets allocated to employer stock were negatively correlated. Further, employees at firms that required employer matching contributions to be allocated to employer stock allocated a larger share of assets to employer stock than those at other firms. The quality of causal evidence presented in this study is low. This low rating means that we are not confident that differences among plan characteristics caused changes in the share of funds allocated to employer stock. Other factors are likely to have contributed.","Number of Fund Choices and Restrictions on Employer Matching Contributions","Employer provided retirement benefits",Employed,,"United States",,,"Behavioral Finance: Retirement Review Protocol"