Enhancing three-dimensional anatomical understanding in complex thoracic surgery: a comparative study of OpVerse and Synapse 3D

Abstract

OBJECTIVES

Virtual reality (VR) technology is increasingly employed in medical settings to provide innovative solutions for complex surgeries. In this study, we introduced and compared OpVerse, a multifunctional new VR platform developed for surgical simulations, with established software Synapse 3D to assess its efficacy in facilitating complex thoracic surgeries.

METHODS

Patient-specific VR digital twin thoracic models were created based on computed tomography scans of 9 patients with large thoracic neoplasms and 4 requiring tracheobronchial reconstruction. Twelve doctors as system testers were enlisted to evaluate the usability and user acceptance of OpVerse and Synapse 3D using the System Usability Scale (SUS) and the Technology Acceptance Model; they provided qualitative feedback through interviews.

RESULTS

OpVerse achieved higher scores than Synapse 3D in SUS (73.3 ± 14.6 vs 53.8 ± 11.6, P = 0.0006), as well as perceived usefulness (4.5 ± 0.4 vs 4.1 ± 0.5, P = 0.0134), perceived ease of use (4.2 ± 0.4 vs 3.8 ± 0.6, P = 0.0364) and attitude towards using and behavioural intention to use (4.6 ± 0.4 vs 3.6 ± 0.7, P = 0.0002) in Technology Acceptance Model, compared to Synapse 3D, indicating enhanced efficiency and user engagement with the new system. Participants favoured OpVerse for its immersive qualities, intuitive interface (particularly rotation and enhanced visual transparency effects) and ability to enhance comprehension of complex 3D anatomical structures.

CONCLUSIONS

OpVerse, our streaming VR simulation platform, enables the manipulation and visualization of patient-specific digital twin thoracic models through features such as rotation, enhanced visual transparency effects and measurement. Preliminary results suggest that OpVerse may offer advantages in terms of immersion, ease of use and understanding of 3D anatomical structures compared to Synapse 3D.

Open in new tabDownload slide

INTRODUCTION

Thoracic surgery’s intricate nature, due to the delicate organs involved, necessitates precise handling and avoidance of iatrogenic harm, particularly in complex cases [1]. While two-dimensional (2D) computed tomography (CT) scans have been crucial for surgical planning, their limitations in conveying spatial relationships can hinder comprehension for those with less medical training. Three-dimensional (3D) imaging overcomes this by providing a superior spatial representation of patient anatomy, facilitating clearer understanding and aiding both surgeons in preoperative planning and patients in making informed decisions.

Initially, the use of 3D reconstruction software, such as Synapse 3D (Fujifilm Medical Co., Ltd, Tokyo, Japan), was popular for this purpose. However, with the gradual rise of virtual reality (VR), this approach has begun to play an increasingly important role in the field of 3D surgical simulation associated with the planning of complex surgical procedures [1].

VR offers an immersive 3D environment for anatomical visualization, enhancing strategic planning and demonstrating clinical benefits like improved preoperative planning, enhanced decision-making and reduced operative time [2, 3]. Additionally, integrating communication technologies with VR facilitates real-time collaboration among multidisciplinary teams, improving patient outcomes and physician efficiency. However, the specific advancements gained by transitioning from 3D models on flat screens to VR experiences remain under-explored in the literature.

In this study, we developed a novel VR surgical simulation system named OpVerse. We conducted a comparative study with Synapse 3D, a commercial 3D reconstruction software package (Fujifilm), to evaluate the usability and effectiveness of both systems in facilitating anatomical understanding. This serves as an initial report on OpVerse, highlighting its potential advantages in surgical planning and education.

PATIENTS AND METHODS

Patients

Thirteen individuals participated in this study. Among them, 9 had large thoracic neoplasms (>5 cm) affecting vital organs such as great vessels, the heart and the airway, or causing nerve compression. The remaining 4 required tracheobronchial reconstruction due to deformative tracheobronchial trees. These patients were enrolled at a single institution between April 2021 and April 2024, with written informed consent obtained from all participants. The National Taiwan University Hospital Research Ethics Committee approved the study (project approval number 202305019RINC, approval date: 23 October 2023).

Testers

Twelve doctors, comprising 3 attending surgeons, 4 senior residents of thoracic surgery and 5 junior residents of general surgery, evaluated the software to assess user experience. All testers provided written informed consent and confirmed no history of cybersickness associated with VR usage. All participants were first-time users of OpVerse and were not involved in any stage of its development process.

Development of OpVerse

With patient consent, a preoperative thin-slice chest CT scan (≤1 mm thickness) was performed without special protocol, which typically takes ∼10 min (Fig. 1A). A senior thoracic surgery resident used Synapse 3D for 3D reconstruction, a process that involves some built-in automation features but still requires significant manual verification and correction (Fig. 1B). This step generally takes ∼1–2 h, depending on the complexity of the case. The accuracy of the final 3D models was verified by an attending thoracic surgeon. A digital twin model mirroring the specific anatomy, including tumour size and location, and vital structures of the patient, was obtained. We exported STL files, commonly used in 3D printing, which represent each organ or system of a patient. These files were subsequently imported into Unity 3D (version 2023.2.15; Unity Technologies, San Francisco, CA, USA), utilized as a game engine for our VR applications and converted into VR-compatible models scaled at a 1:1 ratio. This post-processing step, which applies optimized parameters refined through multiple trials, typically takes ∼0.5–1 h (Fig. 1C). The detailed flowchart of the procedure can be found in Supplementary Material, Fig. 1.

VR digital twin models were displayed within the metaverse, accessed using head-mounted displays (Meta Quest 3, Meta, Menlo Park, CA, USA), enabling users to freely interact with (Fig. 1D) and explore the models from different perspectives in various clinical scenarios (Fig. 2A–C).

We also developed a streaming feature for real-time collaboration and educational purposes, facilitating access through various devices, including mobile phones, personal computers and head-mounted displays. Utilizing web real-time communication, we established a cloud-streaming service, allowing for interactive updates on the server and multicasting to clients, with the audio of the primary user broadcast remotely (Fig. 2D).

Features of OpVerse

Upon receiving the VR headset, users encounter bifurcated interactive content. The right segment displays CT imaging data, accessible through a handheld controller. In contrast, the left presents a 3D thoracic model, using semitransparent and varying colour rendering to differentiate organs and tissues, with pivotal structures annotated. Additionally, we have designed a cross-sectional view of the 3D thoracic model to correlate with the CT images displayed on the right (Fig. 3A). Users can rotate their thoracic model in all directions using a handheld controller equipped with a laser pen to designate target locations. Users can also employ a toggle button to activate or deactivate the 3D model data of the corresponding organ and adjust their viewing angle to transition between transparent and opaque visual presentations.

Furthermore, the distance measurement feature of this system addresses the limitations of traditional 2D imaging modalities, such as CT or magnetic resonance imaging, which are restricted to measuring lengths along the 3 mutually perpendicular axes (X, Y, Z). This feature also facilitates the measurement of curved lengths.

Through these controls, users gain the ability to observe and analyse the anatomical structures of specific cases from every perspective within the VR environment.

Study design

The user experience of OpVerse and Synapse 3D (version 6.7; Fujifilm Medical Co., Ltd) was compared through 3 tasks: counting ribs involved or abutted by the tumour (task 1, Fig. 3B); the maximum length of the thoracic tumour was measured thrice (task 2, Fig. 3C) and measuring the stenotic length of the deformative tracheobronchial tree thrice (task 3, Fig. 3D). Participants performed these tasks using both systems, with the starting system randomly assigned to each participant to avoid bias. Following a 2-min tutorial, participants commenced their assigned 3 tasks. The 1st task, which involved counting the number of ribs abutted by the tumour, was performed once. The 2nd and 3rd tasks, which involved measuring lengths, were repeated 3 times. Participants then completed the System Usability Scale (SUS) and Technology Acceptance Model (TAM) questionnaires, based on a five-point Likert scale. SUS scores were calculated as (X + Y) × 2.5, where X equals the sum of points from odd-numbered questions −5, and Y equals 25 minus the sum of points from even-numbered questions. Following task and questionnaire completion, interviews were conducted to gather subjective evaluations of both systems, guided by a pre-established outline. The study design algorithm is outlined in Fig. 4.

Statistical analysis

The paired t-test was conducted for SUS and TAM score comparison. These analyses were performed using Statistical Analysis System version 9.4. (SAS Institute Inc., Cary, NC, USA). Statistical graphs were generated using GraphPad Prism version 10.0.0 for Windows (GraphPad Software, Boston, MA, USA). All tests were two-sided, and a P-value of <0.05 was considered statistically significant.

RESULTS

Patients

Our study included 13 patients undergoing complex thoracic surgeries. Seven patients had large mediastinal tumours, 2 had Pancoast tumours and 4 underwent tracheobronchial reconstruction using cryopreserved aortic allografts [4] (Table 1). Patients with mediastinal tumours underwent sternotomy with an average operative time of 301.7 ± 127.7 min, experiencing no severe complications (Clavien-Dindo ≤ II). Median intensive care unit and hospital stays were 4 and 10 days, respectively. One patient (case #8) had prolonged stays due to myasthenia gravis-related respiratory failure. For Pancoast tumours, a modified Masaoka incision enabled safe tumour resection, resulting in hospital stays under 10 days and unaffected postoperative activities for both patients. Tracheobronchial reconstruction cases typically exceeded 600 min, except for patient 11, where the operative time was significantly reduced due to localized damage.

User experience

The 12 system testers comprised 4 women and 8 men, with an average age of 33.0 ± 8.2 years (range: 23–55 years) (Supplementary Material, Table S1). Although the initial task completion time was comparable between Synapse 3D (53.8 ± 27.6 s) and OpVerse (52.3 ± 29.1 s), subsequent repetitions of tasks 2 and 3 were faster in OpVerse. Furthermore, OpVerse measurements exhibited less variation (Supplementary Material, Table S2), suggesting greater consistency.

The SUS test results indicate that the average usability score for OpVerse among the 12 doctors was 73.3 ± 14.6, whereas that for the Synapse 3D system averaged 53.8 ± 11.6 (P = 0.0006, Supplementary Material, Table S3 and Fig. 5A).

The TAM assessment demonstrated that OpVerse was rated significantly higher than Synapse 3D in terms of perceived usefulness (4.5 ± 0.4 vs 4.1 ± 0.5, P = 0.0134), perceived ease of use (4.2 ± 0.4 vs 3.8 ± 0.6, P = 0.0364), and attitude towards using and behavioural intention to use (4.6 ± 0.4 vs 3.6 ± 0.7, P = 0.0002) (Supplementary Material, Table S4, Fig. 5B). Detailed SUS and TAM scores for each participant are available in Supplementary Material, Fig. S2 and Supplementary Material, Tables S5 and S6.

Feedback from system testers strongly favoured OpVerse for enhancing 3D anatomical comprehension (Interview outline is provided in Supplementary Material, Table S7). A majority (10/12) found OpVerse superior to Synapse 3D in visuals, usability and immersion, particularly for 3D visualization and multi-angle observation. The enhanced visual transparency effects were most favoured (7/12), aiding understanding of spatial relationships. Additionally, 67% (8/12) emphasized the rotation function’s value in enhancing 3D understanding through multi-angle observation. In terms of interactive features that enhanced engagement, 25% (3/12) appreciated the drag and zoom functions for their flexibility, responsiveness and ease of use, contributing to a more enjoyable user experience.

Most respondents (83%, 10/12) found operating 3D models in VR to be simpler and more intuitive than on a flat screen. Moreover, 75% (9/12) reported that OpVerse offered a stronger sense of immersion and 3D perception, facilitating a clearer understanding of spatial relationships.

However, 17% of respondents (2/12) pointed out some disadvantages of the VR system. Respondent 10 mentioned that the VR system has a high learning cost, requires time to adapt and the necessary equipment can be expensive. Respondent 12 noted that although the VR system provides a seamless interface and model observation, the equipment requirements may limit its widespread adoption. Six respondents suggested improvements in the measurement function, noting that it lacks precision and makes grasping depth and position difficult.

We have provided a comprehensive introductory video about this OpVerse research, encompassing the system development process, functionalities, research design and outcomes. Please refer to the video for a more thorough understanding (Video 1).

DISCUSSION

Three-dimensional imaging enhances understanding of surgical anatomy, particularly complex structures. Commercial software such as Synapse 3D and Mimics, along with open-source options including MITK, 3D Slicer and InVesalius, have minimized unexpected surgical errors in recent years by enabling preoperative simulation [5]. The application of VR for 3D anatomical visualization has been validated as an effective method for surgical planning. However, these 2 modalities—3D reconstruction software and VR simulation systems—have not been directly compared until now. Our newly developed VR system, OpVerse, generates digital twin organ models from specific patient images, offering high immersion. Observing, measuring and manipulating 3D models with this system is straightforward. Additionally, the remote streaming capability of the system, combined with the aforementioned features, allows for its use in medical explanations, multidisciplinary discussions, preoperative planning and education.

In our study, we selected complex mediastinal tumours and tracheal deformities for VR visualization due to their intricate anatomical relationships and critical structures. Mediastinum involves vital organs where any inadvertent injury can have severe consequences. VR models enhance anatomical understanding, increasing surgical confidence and reducing unexpected injuries. Tracheobronchial reconstruction was chosen for its complexity and limited global experience. Our VR system possibly improves understanding of deformed tracheal anatomy, minimizing failure risks. Surgeries with complex anatomy and low tolerance for error are well-suited for preoperative planning using such VR simulation systems. Although we selected these challenging cases as pilot test subjects for this study, the application scope of our system is not limited to these scenarios. For example, common segmentectomy procedures can also benefit from the enhanced stereoscopic depth and immersion provided by our VR system (Video 2).

Compared with 3D reconstruction software, such as Synapse 3D, in this study, respondents reported fewer difficulties using OpVerse. Although 3D reconstruction software offers advanced functions like fusion, extraction and automatic image analyses, untrained operators may face challenges with these functions and interference during basic operations. Difficulties in operating 3D models, particularly in model rotation in Synapse 3D, were also corroborated by the perceived usefulness and perceived ease of use dimension of the TAM questionnaire. The rotation function of OpVerse was frequently cited as a significant advantage, with ∼67% of respondents indicating greater flexibility than Synapse 3D, easing the achievement of desired viewpoints. Furthermore, the enhanced visual transparency effect is a vital design feature of the OpVerse system. Over half of the testers believed that enhanced visual transparency effects facilitated understanding the relative positions of anatomical structures. Owing to superior visual effects, enhanced transparency is another key reason why 80% of testers considered OpVerse superior to Synapse 3D. Additionally, based on scores for attitude towards using and behavioural intention to use in the TAM questionnaire, we believe that OpVerse captures user attention more effectively owing to its superior immersive experience and enjoyable use. The ease of use of OpVerse was also supported by interview results. Many interviewees noted that the OpVerse system is more user-friendly than Synapse 3D for beginners.

OpVerse’s superior performance in SUS and TAM scores compared to Synapse 3D can be attributed to its enhanced immersive experience and user-friendly interface, which includes features such as intuitive controls, efficient model rotation and effective visual transparency. These scores are derived from well-validated and reliable assessment tools, signifying their credibility in evaluating technological usability and acceptance [6, 7]. While this study did not directly assess clinical outcomes, existing research suggests a potential correlation between improved SUS and TAM scores and positive clinical impacts in medical training and patient education. For instance, studies have shown that XR technologies potentially enhance anatomical understanding, improve efficiency in surgical procedures and facilitate surgical training and skill acquisition [8–10]. Given the reliance of surgical procedures on precise anatomical understanding and minimal error tolerance, the immersive nature of VR simulation systems like OpVerse may be particularly valuable in this context.

In our 1st task, using Synapse 3D, 5 out of 12 individuals reported that 4 ribs were in contact with or encased by the tumour. However, using OpVerse, testers reported a maximum of 3 ribs in contact with or encased in the tumour. This discrepancy may be attributed to the simplification of the 3D model during its creation to reduce hardware load. In actual surgery, although the 4th rib came into contact with the tumour, there was no adhesion or invasion. While 3D images or models may not be fully able to predict tumour invasiveness, particularly in borderline cases difficult to predict from original CT images, both OpVerse and Synapse 3D met the original design requirements for accuracy and precision in 3D model display.

VR presents promising opportunities for surgical simulation, yet several challenges persist. The high cost of advanced VR hardware can limit accessibility for some institutions and individuals. While our study did not report instances of cybersickness, its potential impact on users warrants consideration. The current limitations in haptic feedback realism and the steeper learning curve observed in older users represent further areas for improvement. Our research suggests a potential correlation between age and VR adaptation, emphasizing the need for tailored training approaches. Additionally, the physical design of VR headsets, particularly their weight, may pose challenges for patient use. Technical issues can also disrupt training sessions. These factors highlight the necessity for continuous refinement and user-centred design in the development of VR systems for surgical simulation to ensure their widespread adoption and effectiveness in improving surgical training and patient care.

The integration of VR technology into thoracic surgery has revolutionized preoperative planning and surgical training. It provides 3D, interactive and patient-specific anatomical representations that enhance surgical precision, decision-making and outcomes. This marks a significant shift from traditional 2D imaging techniques. Recently, an increasing number of research teams have successfully applied VR technology to thoracic surgeries. However, most of these advances have been published as case reports or case series, lacking comparative studies using existing imaging tools (Supplementary Material, Table S8) [2, 3, 11–18]. A few studies have compared VR with traditional CT imaging, but researchers may have overlooked the comparison between VR and 3D reconstruction software [17, 18]. Prior to the rise of VR, we heavily relied on 3D reconstruction software to aid our understanding of surgical anatomy and preoperative planning [19]. Thus, comprehensive comparative studies are required.

The present study effectively filled this gap. To our knowledge, no previous study has directly compared VR with 3D reconstruction software for application in thoracic surgery. Additionally, this study provided questionnaire surveys and actual operation interviews with not only less experienced postgraduate year residents but also attending thoracic surgeons, contributing to the current literature.

Limitations

Our study has several limitations. First, the lack of multicentre involvement and blinding may introduce bias, as testers were from the same institution where the tool was developed. Although we randomized the order of system usage and included participants from different levels of medical training, the perception of bias remains. Second, while the pilot nature of this study limited the number of testers, we conducted structured interviews to elicit insightful and rich qualitative data, offering valuable insights despite the smaller sample size [20]. Future studies should include testers from different institutions for a more neutral evaluation. Additionally, our data focused on user-based assessments of VR and 3D software usability rather than their direct application in surgical procedures. A prospective, large-numbered, multi-institutional study on perioperative outcomes in real-world settings is needed to establish clinical efficacy.

CONCLUSION

In conclusion, we developed a streaming VR simulation platform, OpVerse, which allows users to manipulate and view patient-specific digital twin thoracic models using functions such as rotation, enhanced visual transparency effects and measurement. Preliminary results suggest that OpVerse may offer advantages in terms of immersion, ease of use and understanding of 3D anatomical structures compared to existing reconstruction software like Synapse 3D. However, further validation is needed. We anticipate that, with additional independent testing, the OpVerse system could be effectively applied to preoperative simulations, multidisciplinary discussions, disease explanations and medical education.

SUPPLEMENTARY MATERIAL

Supplementary material is available at EJCTS online.

FUNDING

This study was supported by the research grants from National Taiwan University Hospital (113-A169, 113-M0029, and 114-M0034) and Taiwan National Science and Technology Council (MOST 113-2321-B-002–044).

Conflict of interest: none declared.

ACKNOWLEDGEMENTS

The authors confirm that OpVerse, the software described in this manuscript, is not currently commercialized or available for sale in any form.

DATA AVAILABILITY

The data underlying this article cannot be shared publicly due to the privacy of individuals that participated in the study. The data will be shared on reasonable request to the corresponding author.

ETHICS APPROVAL

The National Taiwan University Hospital Research Ethics Committee approved the study (project approval number 202305019RINC, approval date: 23 October 2023).

Author contributions

Yu-An Zheng: Data curation; Investigation; Writing—original draft. Yi-Ching Lee: Investigation; Writing—original draft. Jing-Yuan Huang: Data curation; Formal analysis; Investigation. Hsien-Yuan Hsieh: Data curation; Investigation. Yang-Sheng Chen: Investigation; Methodology; Software. Xu-Heng Chiang: Conceptualization; Formal analysis; Project administration; Writing—review & editing. Ping-Hsuan Han: Supervision; Validation. Mong-Wei Lin: Supervision; Validation. Haso-Hsun Hsu: Supervision; Validation. Yi-Ping Hung: Funding acquisition; Methodology; Resources; Software; Supervision. Jin-Shing Chen: Conceptualization; Funding acquisition; Supervision.

Reviewer information

European Journal of Cardio-Thoracic Surgery thanks Maximilian Vorstandlechner, Wouter Bakhuis , Giovanni Luca Carboni and the other, anonymous reviewer(s) for their contribution to the peer review process of this article.

REFERENCES

ABBREVIATIONS

ABBREVIATIONS
 
• 2D

  Two-dimensional

 
• 3D

  Three-dimensional

 
• CT

  Computed tomography

 
• STL

  Stereolithography (file format)

 
• SUS

  System Usability Scale

 
• TAM

  Technology Acceptance Model

 
• VR

  Virtual reality