Citation
Levine, D., Toffel, M., & Johnson, M. (2012). Randomized government safety inspections reduce worker injuries with no detectable job loss. Science, 336(6083), 907-911.
Highlights
- The study’s objective was to examine the effect of random OSHA inspections on firms’ injury-related outcomes and performance.
- To do this, the authors used a regression model to compare changes over time in firms randomly chosen to receive inspections to changes in uninspected firms.
- The study found that the number of injuries and the cost of injuries declined after OSHA inspected a firm. In addition, the study found no evidence that the firms OSHA inspected had lower employment, sales, or credit ratings or were less likely to continue to operate than firms OSHA did not inspect.
- The quality of causal evidence presented in this study is moderate because the study used a well-conducted nonexperimental design. This means we have confidence that the estimated effects are attributable at least in part to OSHA inspections. However, other factors not accounted for in the study could also have contributed to the estimated effects.
Intervention Examined
OSHA Enforcement Activities
Types of and Outcomes
The study examined the effect of a random OSHA inspection on firm-level measures of injury-related outcomes (the number of injuries and total cost of injuries) and firm performance (employment, payroll, sales, survival, and creditworthiness). Firms were designated as having been inspected if they had been randomly selected to have an OSHA inspection and such an inspection had been attempted.
Injury-related outcomes were measured by annual workers’ compensation claims and the dollar value of those claims. Firm performance was measured using sales, employment, payroll, an indicator for whether a firm disappeared from the databases (firm “death”) during the sample period, credit score, and Dun & Bradstreet’s Composite Credit Appraisal score.
Features of the Study
The study analyzed the set of single-establishment firms operating in California from 1996 to 2006 in high-injury industries. Of these firms, some were randomly selected to receive an inspection. However, the authors did not know the set of firms from which the randomly selected firms was drawn, and therefore could not identify the firms that were considered for inspection but not randomly selected.
Instead, the authors created a matched comparison group of firms that had similar characteristics to the randomly inspected firms but had not themselves been inspected. To be considered a match, comparison firms had to be in the same industry and region of California as the randomly inspected firms, be classified as a single-establishment firm, have at least 10 employees, and not have been inspected in the two years preceding the match year.
The study sample consisted of 5,593 firm-year observations from 409 randomly inspected and 409 matched comparison firms, observed from 1996 to 2006. The authors used data on inspections from OSHA’s Management Information System, payroll and injuries from the Uniform Statistical Reporting Plan database (from the Workers’ Compensation Insurance Rating Board), and additional establishment data from Dun & Bradstreet’s compilation of credit ratings and the National Establishment Time-Series database.
The authors used a difference-in-differences model and controlled for year- and firm-level fixed-effects. This type of model compares changes over time in randomly inspected firms to changes over time in uninspected firms. The authors showed that pre-inspection trends in measures of the outcomes were similar for inspected and uninspected firms, demonstrating the validity of this approach.
Findings
- The number of injuries decreased by an average of 9.5 percent after firms received a random OSHA inspection. The cost of injuries (in terms of worker’s compensation claims) also fell by 26 percent after the inspection.
- Firms’ performance, as measured by sales, payroll, employment, creditworthiness, and firm survival, did not change after receiving random OSHA inspections.
Considerations for Interpreting the Findings
Although the firms inspected in this study were randomly selected for inspection, the authors did not know the set from which randomly inspected firms were selected and instead had to select a comparison group from the set of all uninspected firms. This might have included some firms that had initially been selected for inspection but were not actually inspected (inspectors had some discretion over conducting inspections of all randomly selected firms). Because the study groups were not formed randomly, the study should be considered a nonexperimental design.
Given the nonexperimental design, the authors provided ample evidence that the groups being compared in this study were similar before the random inspections were performed. In particular, the two groups of firms had similar values of the outcome variables in the period before the selected firms were inspected as well as similar changes in those values over that period. This gives us confidence that the estimated effects are attributable at least in part to the effect of random inspections.
Causal Evidence Rating
The quality of causal evidence presented in this study is moderate because the study used a well-conducted nonexperimental design. This means we have confidence that the estimated effects are attributable at least in part to the OSHA inspections, although—as with any nonexperimental design—other factors could have contributed to the estimated effect of the inspections.