Absence of conflict of interest.
Citation
Highlights
-
The study’s objective was to evaluate the impact of a Home Healthcare Virtual Simulation Training System (HH-VSTS) on participants’ ability to identify household health and safety hazards and respond appropriately to any concerns.
-
The study was a randomized controlled trial that compared assessment scores for the treatment group who completed the HH-VSTS versus the control group who completed a paper-based training of identical information, but in written hard-copy form. The study assessed participants’ ability to correctly identify a set of hazards, what made each a hazard, and how to respond to each.
-
The study found no differences between the two groups in the ability to correctly identify a hazard or how to address them. However, the study also found that the treatment group was more likely than the control group to incorrectly identify a nonhazard as potentially dangerous.
-
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the HH-VSTS, and not to other factors.
Intervention Examined
Home Healthcare Virtual Simulation Training System (HH-VSTS)
Features of the Intervention
This study authors developed HH-VSTS as a “Serious Game” to teach home students in nursing, occupational therapy, and physical therapy programs, as well as healthcare workers, how to identify and respond to health and safety hazards in clients’ homes. Like other “Serious Games,” HH-VSTS aimed to provide engaging, effective training and education through virtual simulation, to help reinforce the knowledge learned in the classroom to real-world situations. HH-VSTS is an interactive virtual environment representing a two-story house that users must navigate through and identify and address potential hazards. It contains three modules addressing (1) electrical/fire/burn hazards, (2) slip/trip/lift hazards, and (3) environmental hazards. After completing a walk-through video and tutorial, users navigate through the virtual house, identifying hazards and potential solutions.
Features of the Study
The study was a randomized controlled trial, with treatment and control status randomly assigned to 22 different training times, using a block randomization schedule. Participants then selected the training time that worked best for their schedule and received whichever training type was assigned to that timeslot. The treatment group consisted of 39 participants who took the virtual training (HH-VSTS), of whom 38 completed the assessment module. The control group was comprised of 36 participants who completed a paper-based version of the training, all of whom completed the assessment. The study took place at community home healthcare agencies and a university. About 87% of participants were female, and about 81% were white. Slightly over half of participants were healthcare students, while the others were home healthcare workers who had been employed in the industry for an average of eight years. There were no significant demographic differences between the intervention and comparison groups. The authors compared the scores of the intervention and comparison groups and reported results in percentages.
Findings
Health and Safety:
-
The study found no significant differences between the two groups in correctly identifying hazards, identifying what makes something a hazard, or responding to what to do about a hazard.
-
The study found that the treatment group over-identified nonhazards as potentially dangerous, compared to the control group. Specifically, the treatment group correctly rejected 48% of nonhazards while the control group correctly rejected a higher share of nonhazards, 61%.
Considerations for Interpreting the Findings
Although the study was a well-implemented randomized controlled trial with low attrition, it had a small sample of only 74 participants. Additionally, the authors examined multiple related impacts on outcomes in the Health and Safety domain. Performing multiple statistical impacts on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect a true program effect. The study also found that participants with ‘moderate’ to ‘high’ self-reported computer use or gaming experience reported greater ease in navigating the simulated house than those with ‘low’ amounts of self-reported computer use. Although there were no other significant differences in usability reported between these intervention groups, a participant’s amount of prior computer experience could have contributed to the lack of significant differences in outcomes between the intervention and control groups. Finally, although it seems that participants did not know which training type would be offered when they selected which training time to attend, if any participants did have that information in advance, then preexisting differences between the groups and not the training itself could potentially explain the observed differences in outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the HH-VSTS, and not to other factors.