Absence of conflict of interest.
Citation
Highlights
-
The study's objective was to examine the impact of using a risk management process on the fire department’s average monthly crash rate and crash rate trends, as well as on crash rates and trends for a specific type of call, “Code 3 responses,” when lights and sirens are activated. The authors investigated similar research questions for two other fire departments, the profiles of which can be found here:
-
The study used an interrupted time series design to compare the fire department’s crash rates before and after the risk management process was implemented. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes.
-
The study found a statistically significant decline in the average monthly crash rate overall and for Code 3 responses.
-
The study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.
Intervention Examined
Risk Management
Features of the Intervention
Emergency service vehicle crashes are one of the leading causes of death for firefighters. Risk management aims to proactively identify and reduce occupational risks and hazards. Risk management has been used in many different occupational settings, including in fire departments to reduce occupational injury and deaths.
In this study, firefighters and department administrators formed a risk management team to work with the study authors to conduct a formal risk assessment for emergency service vehicles. Over four to six meetings, the process involved reviewing emergency services vehicles crash data, conducting a formal risk assessment by creating a register of and prioritizing common crash types and risks, cataloguing existing risk reduction initiatives, and identifying additional risk reduction initiatives. In the second phase, the department implemented additional risk reduction initiatives identified during Phase 1. Fire Department A revised its policy for “lights and siren” responses (“Code 3”), revised its standard operating procedure for backing up, installed additional cameras on some ambulances, and increased remedial driver training.
The intervention is designed to serve any emergency service provider with their own vehicles such as fire departments.
Features of the Study
The study used an interrupted time series design to compare the department’s crash rates before and after the risk management process was implemented. The department reported 28 months of data prior to the risk management process and 23 months after. The study uses data from administrative records, fire department crash reports, and emergency fire service call volumes. Crash records are generated from department reports submitted when crashes involving any fire department vehicles result in damage to a department vehicle or to a civilian vehicle or property, in accordance with the fire departments rules and regulations. Crash records include an indicator denoting whether or not lights and sirens were activated (Code 3 response).
The study sample is one, large, urban U.S. fire department. This fire department served more than 1 million residents in a dense urban environment, had over 1,000 career staff and more than 200 emergency service vehicles at more than 50 fire stations.
The fully specified model included an overall time trend and the number of months pre- or post-intervention, controlling for the average crash rate prior to the intervention, and correcting standard errors to account for correlation of observations over time.
Findings
Health and Safety
-
The study found a statistically significant decline in the average monthly crash rate following the implementation of risk management.
-
In the month immediately after the risk management process, the crash rate was not significantly different than before.
-
The overall crash rate trend was not significantly different than before.
-
The study found a statistically significant decline in the average monthly crash rate for Code 3 responses to emergency calls with lights and sirens active.
-
In the month immediately after the intervention, the crash rate for Code 3 responses was not significantly different.
-
The study found a statistically significant decline over time in the Code 3 crash rate trend after the intervention.
Considerations for Interpreting the Findings
The authors compared the fire department’s outcomes measured before and after the department participated in the intervention. For this type of design, the authors must introduce the intervention more than once or in multiple settings at different time points to rule out the possibility that something else changed at the same time and could explain the study results.
The fire department and its staff could have anticipated the beginning of the risk management process and begun adjusting their behavior in advance.
The study authors estimated multiple related impacts on outcomes related to health and safety. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will appear statistically significant purely by chance and not because they reflect the program’s effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains could be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors only examine trends for one department and only introduced the intervention once. This means we are not confident that the estimated effects are attributable to the risk management process; other factors are likely to have contributed.