Absence of conflict of interest.
Citation
Hoebbel, C., Bauerle, T., Macdonald, B., & Mallett, L. (2015). Assessing the effects of virtual emergency training on mine rescue team efficacy. Paper presented at the meeting of the National Institute for Occupational Safety and Health, Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) in Pittsburgh, PA.
Highlights
- The study’s objective was to examine the impact of virtual mine rescue training on mine rescue process efficacy.
- Using survey data, the authors used an interrupted time series design to compare mine rescue team members’ self-reported process efficacy before and after virtual mine rescue training.
- The study found that the virtual mine rescue training was significantly associated with an increase in process efficacy.
- The quality of causal evidence presented in this report is low because the authors did not observe outcomes for multiple periods before and after the intervention. This means we are not confident that the estimated effects are attributable to the mine rescue training; other factors are likely to have contributed.
Features of the Study
The nonexperimental study was conducted at the National Institute for Occupational Safety and Health’s (NIOSH) Office of Mine Safety and Health Research’s Virtual Immersion and Simulation Laboratory (VISLab) in Pittsburgh, Pennsylvania. VISLab provides a virtual reality simulation environment where mine workers can measure and develop their knowledge, skills, abilities, and other attributes (KSAOs) required for successful emergency responses.
Study participants included 70 miners from ten teams across Pennsylvania and West Virginia (six company teams, three composite state teams, and one university team). Prior to the training, participants completed a short orientation to the virtual environment and attended a briefing where they were assigned different team roles. During the mine rescue training simulation, participants experienced one of two simulated training modules (lasting 60 to 90 minutes) where they virtually navigated through the mine using simulated items and tools, while remaining in contact with their briefing officers and teammates in the mine. After the training, participants were debriefed and reviewed the virtual session. The authors assessed three outcomes using a survey:
- Process efficacy is a team’s belief that they can successfully engage in the processes necessary to complete an outcome. Process efficacy was comprised of two frameworks: (1) “Big Five” framework including team leadership, mutual performance monitoring, backup behavior, adaptability, and team orientation; and (2) “Coordination” framework including the team's shared mental models, closed-loop communication, and mutual trust;
- Team familiarity is how often team members see each other at work or outside of work and how often training across team roles occurs; and
- Training climate is the degree to which workplace supports exist and represent the mine sites’ values.
Participants completed the survey three times: before briefing, immediately after the simulation, and after debriefing. Using self-reported survey data, the authors conducted statistical analyses to examine differences in process efficacy before and after the virtual training and to identify the relationships between process efficacy, team familiarity, and training climate.
Findings
Health and safety
- The study found that the virtual mine rescue training was significantly associated with an improvement in overall process efficacy.
- When examining the Big Five framework, the study found that being on a team with high familiarity and/or a supportive training climate was significantly associated with high process efficacy.
- The study also found that being on a team with high familiarity was significantly related to greater efficacy gains for the Coordination framework; however, no significant association was found with training climate.
Considerations for Interpreting the Findings
All data were collected using three surveys over a four-hour timeframe. The authors compared the process efficacy of participants measured once before and twice after they participated in the virtual mine rescue training. CLEAR’s guidelines require that the authors observe outcomes for multiple periods before the intervention to rule out the possibility that participants had increasing or decreasing trends in the outcomes examined before enrollment in the training. Without knowing the trends before enrollment, we cannot rule this out. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because authors do not observe outcomes for multiple periods before or after the intervention. This means we are not confident that the estimated effects are attributable to the virtual mine rescue training; other factors are likely to have contributed.