Clinical Readiness: Can Providers Learn to Perform Lower Leg Fasciotomy Through a Tablet-based Augmented Reality Surgical Training Environment?

ABSTRACT

Introduction

The uses of on-demand, interactive tablet-based surgical training environments are of interest as potential resources for both the acquisition and maintenance of rarely performed, critical procedures for expeditionary surgical care. This study examined the effectiveness of a tablet-based augmented reality (AR) procedural training environment for lower leg fasciotomy with a cohort of novice surgical trainees in (1) procedural knowledge, (2) tablet-based procedural skills, (3) tablet-based procedural time, and (4) procedural performance on a cadaver. We hypothesized that engaging with the AR procedural training would increase procedural knowledge and tablet-based skills and procedural time. We hypothesized that the tablet-based AR training environment would be insufficient to acquire the ability to perform lower leg fasciotomy on a cadaver.

Materials and Methods

This study was approved as exempt by the Institutional Review Board at USU. Surgical interns, sub-interns, and independent duty corpsman (n = 30) with no prior lower leg fasciotomy experience voluntarily participated. Tablet-based training activities included pre-training assessment, engagement with instruction, interactive procedural practice, and post-training assessment. Tablet-based knowledge assessment included 17 multiple choice questions covering concepts, reasoning, and judgment associated with the procedure. Tablet-based procedural completion and time were assessed within the training environment. Within 1 week of completing the tablet activities, participants were assessed by fellowship-trained trauma surgeons while performing cadaver-based lower leg fasciotomy. Statistical analysis included paired t-tests and effect size (Cohen’s d). Statistical significance was set at P < .05.

Results

Tablet-based AR procedural training significantly improved procedural knowledge (P < .001), tablet-based procedural skills (P < .001), and reduced tablet-based procedural time (P < .002). Effect sizes were very large for tablet-based procedural knowledge (d = 1.75) and skills (d = 3.2) and small (d = 0.42) for procedural time. There were no significant effects of procedural knowledge, tablet-based procedural skills, or time on cadaver-based performance. No participant was able to accurately and independently complete lower leg fasciotomy procedure on a cadaver.

Conclusions

Tablet-based AR procedural training improved procedural knowledge and tablet-based skills; however, those gains did not transfer to the ability to perform the procedure on a cadaver. The tablet’s limited AR interface did not support the acquisition of requisite surgical technique, tissue handling, and decision-making in novice surgical trainees. Experienced surgeons may have different outcomes because their mature understanding of surgical constructs would allow extrapolation of abilities to other procedural contexts. Further investigation of the tablet-based training environments for surgical care is necessary before distributing such resources to support clinical readiness.

INTRODUCTION

The movement toward technology centered solutions to advance the acquisition of human performance capabilities or to further enhance acquired performance abilities using these same technologies as supporting resources is an ongoing conundrum for preparing military medical providers to acquire and maintain clinical abilities that are rarely, yet critically required for combat casualty care. The temptation is to infer that the convergence of rapid advances in robotics, information technology, and artificial intelligence (AI) into integrated human–machine augmented performance systems for tactical military operational demands could, similarly, apply to the development of technology augmented clinical performance systems, be they for the processes of performance acquisition, maintenance, or augmentation. The temptation to consider the development of realistic but safe synthetic training and practice environments for rarely performed critical combat casualty care procedures is appealing for convenience, consistency, and economic value for broadly distributed training implementation. All the more enticing is the potential development of augmented human–machine teams to mitigate the potential performance impacts of unmastered human performance abilities that are vulnerable to the psychophysiological stressor on the cognitive and kinesthetic performance components concomitant with combat casualty care. Although these concepts are appealing and scientific advancements suggest their potential realization in some human performance constructs, high-stakes performance constructs merit independent, comprehensive analyses to characterize the value of any technology-based platform for the purposes of human performance acquisition, maintenance, or augmentation.¹

The evaluation of these types of integrated, technology platforms for medical and surgical applications is critical before broad scale implementation because the potential adverse outcomes must be equally considered with potential benefits. The fastest growing integration of these technologies is in medical schools, ostensibly the seminal ground for the acquisition of procedural abilities. For example, the rapid integration of interactive augmented reality (AR) platforms for learning human anatomy, such as the Anatomage, has replaced hands-on engagement with cadaveric specimens in many medical schools.² There appears to be significant value for medical students in developing their foundational understanding of human anatomical principles, structures, normative configurations, and inter-relationships.^3–5 However, the traditional approach to learning anatomy through cadaveric dissection also includes several critical performance factors that are sacrificed through the use of this technology-based instruction: (1) Direct observation of anatomical variances within and between cadaveric specimens; (2) development of tactile sensitivities to tissues structures that are critical for all medical specialties (but especially surgical specialties); (3) development of essential dissection skills including the uses of specific surgical instruments; and (4) recognition of pathological conditions through direct observation within and between patients. These factors are necessary for the development of critical abilities associated with clinical reasoning, judgment and decision-making during clinical practice, and disadvantages those who enter their respective residencies without having developed those abilities within the context of also learning human anatomy.^6,7 They are also critical for laying the foundational understanding of tissue and the interaction between surgical instruments and tissues, which are critical for surgical specialties. Additionally, surgery is a highly visual and tactile specialty that demands mastery of the three-dimensional arrangement of anatomical structures and their inter-relationships to identify and work within fundamental principles of clinical practice.⁸

There are many challenges associated with the uses of virtual reality (VR), AR, AI, and other advanced technologies to provide augmented performance support resources for surgical and procedural demands, and these also apply to procedural training or practice environments.⁹ For clinical skills training, AR mixes digital information with real-world stimuli to create a mixed-reality (MR) environment that allows users to interact with virtual information using tactile feedback in a way that offers a significantly better use experience than VR alone.^10–12 However, Mackenzie et al. evaluated outcomes from VR, AR, MR, and haptic interface platforms for surgical performance development in open trauma and emergency surgery patient care contexts and found that there was inadequate evidence that VR, MR, AR, or haptic interfaces can facilitate training for open trauma surgery or replace cadavers.¹³

Notwithstanding these challenges, there remains a high demand signal for technology systems to support the acquisition and retention of combat casualty care abilities and a significant push in military medicine toward leveraging emerging technologies to support the development of medical expertise across the spectrum of clinical providers from medical school through continuing medical education for experience clinicians.¹⁴ In their meta-analysis of reported procedural performance outcomes resulting from the uses of VR, AR, AI, and other immersive training platforms, Williams et al. found neither continuity nor consensus agreement between the identified studies for performance acquisition, maintenance, and augmented applications.¹⁵ Their conclusions assert the need for more performance outcomes studies to examine the potential benefits and challenges associated with leveraging these technologies for the acquisition of procedural abilities. In particular, they cited the need for studies that examine the benefits and challenges of technology-enabled support resources at different starting positions on a learning curve to accurately and comprehensively determine their potential value for performance acquisition, maintenance, and augmentation.

Questions remain about the potential value of integrated technology-enabled environments for the acquisition, maintenance, and augmentation of clinical abilities that transfer to actual patient care. Given that procedural skills performance acquisition begins during medical school and the rapid uptake of these technologies for early training is becoming ubiquitous, the value of a technology-enabled learning environment for procedural performance acquisition would be best performed with relative novices. The purpose of this study was to determine the extent to which an AR procedural training environment for lower leg fasciotomy supported the acquisition of procedural abilities (cognitive and psychomotor skills) that transferred to procedural performance on a cadaver.

MATERIALS AND METHODS

Study Design

This study was approved by the Institutional Review Board at USU as an exempt study. A purposive sample of 30 novice surgical trainees determined by a priori calculation with 95% power volunteered to participate in the study from Walter Reed National Military Medical Center Department of General Surgery (n = 20) and Naval Medical Center San Diego Department of General Surgery and Surface Warfare Medical Institute West (n = 10). Novice surgical trainees had no prior fasciotomy experience and were either surgical interns, surgical sub-interns, or U.S. Navy independent duty corpsman (IDC) students. Novice surgeons were selected specifically to examine the effectiveness of the tablet-based AR environment for surgical training at the lower end of the provider capability spectrum (without extensive surgical experience and well-developed tissue handling abilities).

We examined the extent to which a tablet-based AR surgical training environment (Sharp Vision Software, LLC) supports the acquisition of lower leg fasciotomy procedural abilities for novice surgical trainees in four performance areas: (1) Procedural knowledge, (2) tablet-based procedural skills, (3) tablet-based procedural time, and (4) cadaver-based procedural performance. We hypothesized that tablet-based AR training would improve procedural knowledge and tablet-based AR skills and procedural time but would be insufficient to acquire the ability to perform the lower leg fasciotomy procedure on the cadaver to an established competency standard of accurate and independent implementation of all procedural components.

Procedural Context

Compartment syndrome is a surgical emergency that requires fasciotomy as definitive treatment to avoid potential loss of life or limb.¹⁶ Fasciotomy facilitates surgical release of the fascia and decompression of elevated intra-compartmental pressures that result from enclosed and pathologically congested osteo-fascial spaces.¹⁷ Approximately 7 to 11% of civilian tibia fractures result in compartment syndrome, with an even greater prevalence in military trauma with high energy mechanisms of which extremity injuries account for nearly two-thirds of combat wounds.^18–20 The risk of irreversible tissue necrosis markedly increases without full decompression via fasciotomy within 6 hours of developing compartment syndrome.^21–24 However, the opportunity to perform fasciotomies in the management of traumatic injuries is limited for most general surgeons and even more limited for surgical trainees due to reduced training opportunities for open surgical procedures.^25–27 These performance deficits may be mitigated through cadaver-based training from fellowship-trained trauma surgeons; however, resource limitations associated with routine cadaver-based instruction restrict this method for on-demand training or in forward deployed settings.^28–30

Variables and Measurement

Study activities and sequence included (1) pre-training assessment of fasciotomy procedural knowledge and procedural skills via tablet-based AR testing; (2) procedural instruction and practice via tablet-based AR training modules; (3) post-training assessment of lower leg fasciotomy knowledge, skills, and procedural completion via tablet-based AR testing; (4) post-training assessment of procedural performance abilities on a cadaver.

Tablet-based knowledge assessment included 17 multiple choice questions specific to lower leg fasciotomy including procedural sequence, relevant anatomy, landmark identification, incision, dissection, and lateral versus medial approaches (Supplementary S1). The total possible knowledge score was 17. Tablet-based AR procedural skills assessment examined the accurate completion of procedural steps for lower leg fasciotomy performed with tablet stylet on a simulated leg as shown in Figs 1 and 2. Assessment markers within the AR training environment included complete decompression of the four compartments without injuring neurovascular structures. The total possible procedural skills score within the training environment was 100, and the procedural time was measured as the total time to complete the AR procedure. Each participant was able to engage with the training activities without time restrictions and could choose when they were ready to move to the assessment activities.

Cadaver-based procedural skills assessment was completed within 1 week of the participant completing all tablet-based activities. Participants did not to engage with additional procedural resources during that time. Cadaver-based procedural assessment examined independent completion of a lower leg fasciotomy procedure on a fresh-frozen cadaver. Procedural performance measures included identification of anatomical structures, correct procedural sequencing, accurate locations of incisions, completeness of compartment decompression, protection of neurovascular structures, and avoidance of iatrogenic injury. Critical procedural steps were specified as those that if not performed correctly would lead to patient morbidity or mortality. The total possible score for cadaver-based procedural skill assessment was 100, with a score of 90/100 indicating an independent and accurate performance of procedural steps. Assessment instrumentation and evaluated procedural steps in the cadaver-based performance were validated as part of the American College of Surgeons course Advanced Surgical Skills for Exposure in Trauma Plus (ASSET+) and the Joint Trauma System’s clinical readiness program.²⁹

Pre- and post-training assessment data for the tablet-based AR testing modules were collected within the training environment and exported for analysis. Cadaver-based procedural assessments were completed real-time by fellowship-trained trauma surgeons who are certified ASSET+ instructors to ensure validity and reliability of scoring.^29,30 Each study participant was paired one-to-one with an assessor, who completed the assessments using software (Ace Surgery, Ganasa, LLC) to record the performance data to a secure server, and subsequently exported for analysis.

Statistical Analysis

Analyses of pre-/post-training performance outcomes were completed using paired t-tests and effect size (Cohen’s d) calculations. Statistical significance was set at P < .05. Statistics were performed using IBM SPSS Statistics.

RESULTS

Descriptive statistics, effect size calculations, and t-test statistical significance for each variable are shown in Table I. The distribution of the participants included 66.67% (20/30) males, 33.33% (10/30) females, 26.67% (8/30) surgical interns, 33.33% (10/30) surgical sub-interns, and 40.00% (12/30) IDCs. Tablet-based AR training resulted in a significant improvement (P < .001; very large effect size) in procedural knowledge, tablet-based procedural skills, and reduced time to complete the AR procedure (P < .002; small effect size). Importantly, the learning demonstrated within the tablet-based AR environment did not transfer to the ability to perform a lower leg fasciotomy procedure on a cadaver. None of the participants were able to perform the lower leg fasciotomy, with both the total procedural score (60.96 ± 24.44) and critical task score (40.81 ± 28.58) significantly below the minimum score of 90/100 indicating competent performance (Fig. 3). The results supported our hypothesis that the tablet-based AR training environment would not transfer to applied surgical performance on the cadaver.

DISCUSSION

We examined the impact of a tablet-based AR training application for the acquisition of lower leg fasciotomy procedural abilities in a cohort of novice surgical trainees in four performance areas: (1) Procedural knowledge, (2) tablet-based procedural skills, (3) tablet-based procedural time, and (4) cadaver-based procedural performance. The study outcomes demonstrated improvements in tablet-based AR procedural knowledge, skills, and time to complete the lower leg fasciotomy procedure; however, those gains did not transfer to independent and accurate (90/100) cadaver-based performance. These study data suggest that this AR training modality was insufficient for acquiring the surgical skills required for applied procedural performance in the context of patient care.

The most likely explanation for these outcomes is that the surgical novices lack the prerequisite surgical abilities needed to transfer the tablet-acquired AR learning to the application of real surgical instruments, real anatomy, and cadaver tissue. The limitations of the tablet-based AR interface are likely insufficient for learners to be able to connect the learning gained through the training environment to actual operative experience because they lack the foundational anatomical understanding, surgical instrumentation, tissue handling, and surgical techniques that would allow them to connect the new learning to previously acquired understanding of surgical processes.⁶ This is an important outcome for the development of on-demand training resources for physicians and surgeons deployed to expeditionary and far forward sites, who may be required to perform particular procedures without acquired abilities. There is likely an equivalent potential need for this type of training in prolonged casualty care settings, where an IDC or paramedic level provider without sufficient experience or training to perform necessary procedures may be the highest-level provider at point of injury.

As a platform for procedural training, the tablet-based AR environment did not facilitate acquisition of fasciotomy procedural skills for operative performance with surgically inexperienced users. More experienced surgeons may be able to transfer tablet-acquired AR learning to an applied surgical context (cadaver or clinical) because they have developed foundational surgical skills capabilities.⁶ Experienced surgeons have performance benefits acquired from a greater number and diversity of operative experiences over that time, which enables them to adaptively accommodate a range of operative circumstances within the domain.⁷ If surgeons have mastered and maintained their surgical abilities, they will be able to apply them to new procedural contexts without needing to re-learn foundational procedural techniques (e.g., incision placement, blunt or sharp dissection, etc.).³¹

Therefore the performance improvements demonstrated within the AR environment may be more readily integrated and applied to patient care by experienced surgeons. Meta-analyses of similar studies suggest that technology-based environments for the acquisition and maintenance of procedural abilities are more impactful for learners with more practical experience in their respective performance domains.¹⁵

For more experienced learners, tablet-based AR procedural training environment may be effective for maintaining previously acquired abilities or even expansion of practice to include procedures that are not typically performed as part of their routine surgical practice. In that capacity, the training platform examined for this study could provide an on-demand resource for sustaining surgical abilities in the setting of trauma care, especially in rural, remote, expeditionary, combat, or global health contexts. However, the study outcomes also suggest that tablet-based AR procedural training environment assisted in the development of procedural knowledge, which may enable greater participation of inexperienced learners during faculty-supervised operative training in either laboratory or patient care settings, thereby leveraging the time available during a procedure to develop critical technique, procedural decision-making, reasoning, and judgment.

The outcomes of this study are limited to the features and performance of the specific tablet-based AR procedural training environment for lower leg fasciotomy; however, the constraints of the technologic and graphical user interface represent a limitation for tablet-based procedural trainers in general. Nonetheless, tablet-based training environments remain of interest for on-demand clinical training because they provide a compact, accessible, and cost-effective modality compared to other hands-on training options, especially in expeditionary settings. The outcomes of this study demonstrate that for tablet-based AR training environments tied to critical procedural care, it is essential to identify the strengths and limitations of the training platform for all levels of providers. Although there is a strong desire for on-demand tablet-based procedural instruction, the effectiveness of these training platforms must be determined before they are promoted for high-stakes learning requirements. Training environments built on a backbone of integrated technologies may continue to improve to include elements that facilitate the development of kinesthetic and psychomotor abilities associated with procedural performance, but in the interim these types of training platforms would best be considered as adjunctive to cadaver-based and live tissue training.

Limitations

All procedural assessors were fellowship-trained trauma surgeons who are certified ASSET+ instructors; however, potential variation may exist between their perceptions and judgments of learner performance. Additionally, the performance of procedural abilities with cadavers may not transfer to applied surgical practice, as the operative environment has significantly more complex patient and environmental challenges. It logically follows that the extent to which the learners’ procedural abilities would transfer to operative patient care would likely be significantly worse.

CONCLUSION

The tablet-based AR procedural training environment for lower leg fasciotomy supported the acquisition of procedural knowledge for novice surgical trainees. However, the environment did not support the development of procedural abilities to independently and accurately perform lower leg fasciotomy procedure on a cadaver. Further investigation is needed to determine the value of the tablet-based AR procedural training environment as a resource for experienced surgeons to maintain procedural currency or expand their practice to accommodate patients in a trauma setting. In this regard, a tablet-based AR procedural training environment might have value as an adjunctive tool for clinical skills sustainment for military and civilian trauma surgeons.³² This type of training environment may also benefit graduate medical education as an advanced organizer of procedural requirements ahead of operative patient care settings, thereby enabling greater trainee participation and engagement during supervised procedures.

ACKNOWLEDGMENTS

None declared.

CLINICAL TRIAL REGISTRATION

Not applicable.

INSTITUTIONAL REVIEW BOARD (HUMAN SUBJECTS)

This study was approved by the Institutional Review Board at USU as an exempt study.

INSTITUTIONAL ANIMAL CARE AND USE COMMITTEE

Not applicable.

INDIVIDUAL AUTHOR CONTRIBUTIONS

K.W. assisted with the study design, data collection, and manuscript preparation. Mar.B. assisted with the study design, data collection, app facilitation, and manuscript preparation. Mat.B., B.F., and R.D. assisted with the data collection and manuscript review. E.W. assisted with the data collection, app facilitation, and manuscript preparation. P.A. assisted with the study design, data collection, data analyses, manuscript preparation, and project oversight.

INSTITUTIONAL CLEARANCE

Institutional clearance was approved.

SUPPLEMENTARY MATERIAL

SUPPLEMENTARY MATERIAL is available at Military Medicine online.

FUNDING

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CONFLICT OF INTEREST STATEMENT

The authors report no proprietary or commercial interest in any product mentioned or concept discussed in this article.

DATA AVAILABILITY

The data that support the findings of this study are available on request from the corresponding author.

REFERENCES

Author notes