Absence of conflict of interest
Citation
Highlights
-
The study's objective was to examine the impact of using a risk management process on a fire department’s average monthly crash rate and crash rate trends. The authors investigated similar research questions for two other fire departments, the profiles of which can be found here:
-
The study used an interrupted time series design to compare the fire department’s crash rates before and after the risk management process was implemented. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes.
-
The study found that no statistically significant differences between the department's crash rates and crash rate trends after the risk management procedures were implemented.
-
The study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.
Intervention Examined
Risk Management
Features of the Intervention
Emergency service vehicle crashes are one of the leading causes of death for firefighters. Risk management aims to proactively identify and reduce occupational risks and hazards. Risk management has been used in many different occupational settings, including in fire departments to reduce occupational injury and deaths.
In this study, firefighters and department administrators formed a risk management team to work with the study authors to conduct a formal risk assessment for emergency service vehicles. Over four to six meetings, the process involved reviewing emergency services vehicles crash data, conducting a formal risk assessment by creating a register of and prioritizing common crash types and risks, cataloguing existing risk reduction initiatives, and identifying additional risk reduction initiatives. In the second phase, the department implemented additional risk reduction initiatives identified during Phase 1. Fire Department B specifically focused on recent deer collisions, distributing maps of high-risk areas for these crashes, updated the driver training curriculum, instituted training for using the bay doors, and installed units on some department vehicles to identify high-risk driving behaviors.
The intervention is designed to serve any emergency service provider with their own vehicles such as fire departments.
Features of the Study
The study used an interrupted time series design to compare the department’s crash rates before and after the risk management process was implemented. The department reported 27 months of data before the risk management process and 24 months after. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes. Crash records are generated from department reports submitted when crashes involving any fire department vehicles result in damage to a department vehicle or to a civilian vehicle or property, in accordance with the fire departments rules and regulations.
The study sample is one suburban county fire department in the U.S. This fire department relied on career and volunteer staff and 50 to 100 emergency service vehicles to serve a suburban county with fewer than 500,000 people.
The fully specified model included an overall time trend and the number of months pre- or post-intervention, controlling for the average crash rate prior to the intervention, and correcting standard errors to account for correlation of observations over time.
Findings
Health and Safety
-
The study found no statistically significant change in the department’s average monthly crash rate after the intervention was implemented.
-
Additionally, the study found no statistically significant change in the crash rate in the month immediately after the intervention.
-
The study found no statistically significant relationship between the crash rate trend and the risk management initiatives.
Considerations for Interpreting the Findings
The authors compared the fire department’s outcomes measured before and after the department participated in the intervention. For this type of design, the authors must introduce the intervention more than once (or in different settings at different times) to rule out the possibility that something else changed at the same time and could explain the study results.
Additionally, the fire department and its staff could have anticipated the beginning of the risk management process and begun adjusting their behavior in advance.
The study authors estimated multiple related impacts on outcomes related to health and safety. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will appear statistically significant purely by chance and not because they reflect the program’s effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains could be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors only examine trends for one department and only introduce the intervention once. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.