Absence of conflict of interest.
Citation
Highlights
-
The study's objective was to examine the impact of using a risk management process on the fire department’s average monthly crash rate and crash rate trends overall, as well as on rates and trends for crashes that the department deemed preventable. The authors investigated similar research questions for two other fire departments, the profiles of which can be found here:
-
The study used an interrupted time series design to compare the fire department’s crash rates before and after the risk management process was implemented. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes.
-
The study found that the crash rate was significantly lower in the month immediately after the intervention. The study found no statistically significant difference on the overall crash rate trends, or crash rates for preventable crashes.
-
The study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.
Intervention Examined
Risk Management
Features of the Intervention
Emergency service vehicle crashes are one of the leading causes of death for firefighters. Risk management aims to proactively identify and reduce occupational risks and hazards. Risk management has been used in many different occupational settings, including in fire departments to reduce occupational injury and deaths.
In this study, firefighters and department administrators formed a risk management team to work with the study authors to conduct a formal risk assessment for emergency service vehicles. Over four to six meetings, the process involved reviewing emergency services vehicles crash data, conducting a formal risk assessment by creating a register of and prioritizing common crash types and risks, cataloguing existing risk reduction initiatives, and identifying additional risk reduction initiatives. In the second phase, the department implemented additional risk reduction initiatives identified during Phase 1. Fire Department C updated their driver training program, installed monitoring units on some vehicles to identify high-risk driving behaviors, and used data from the monitoring units to coach drivers.
The intervention is designed to serve any emergency service provider with their own vehicles such as fire departments.
Features of the Study
The study used an interrupted time series design to compare the department’s crash rates before and after the risk management process was implemented. The department reported 29 months of data before the risk management process and 22 months after. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes. Crash records are generated from department reports submitted when crashes involving any fire department vehicles result in damage to a department vehicle or to a civilian vehicle or property, in accordance with the fire departments rules and regulations. Preventable crashes are those deemed to be caused by the driver's actions or that the driver could have avoided through reasonable actions, as determined by the department’s crash review panel.
The study sample is one large suburban county fire department in the U.S. This fire department served more than 1 million residents in a suburban county over 2,000 square miles. The fire department employed more than 1,000 career staff at more than 50 fire station, and had between 100 to 200 emergency service vehicles.
The fully specified model included an overall time trend and the number of months pre- or post-intervention, controlling for the average crash rate prior to the intervention, and correcting standard errors to account for correlation of observations over time.
Findings
Health and Safety
-
The study found no statistically significant difference in the average monthly crash rate after the risk management process.
-
In the month immediately after the risk management process, the study found a statistically significantly lower crash rate.
-
The overall crash rate trend was not significantly different than before the risk management intervention.
-
The study found that no statistically significant differences in the outcomes measuring preventable crashes: that is, the average monthly preventable crash rate, overall preventable crash rate trend, and preventable crash rate in the month immediately after the intervention were not statistically significantly different.
Considerations for Interpreting the Findings
The authors compared the fire department’s outcomes measured before and after the department participated in the intervention. For this type of design, the authors must introduce the intervention more than once (or in multiple settings at different times) to rule out the possibility that something else changed at the same time and could explain the study results.
Additionally, the fire department and its staff could have anticipated the beginning of the risk management process and begun adjusting their behavior in advance.
The study authors estimated multiple related impacts on outcomes related to health and safety. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will appear statistically significant purely by chance and not because they reflect the program’s effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains could be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors only examine trends for one department and only introduce the intervention once. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.