EACVI SIMULATOR-online study: evaluation of transoesophageal echocardiography knowledge and skills of young cardiologists

Abstract

Aims

To assess the level of transesophageal echocardiography (TOE) knowledge and skills of young cardiologists.

Methods and results

A European Association of Cardiovascular Imaging (EACVI) online study using the first fully virtual simulation-based software was conducted in two periods (9–12 December 2021 and 10–13 April 2022). All young cardiologists eligible to participate (<40 years) across the world were invited to participate. After a short survey, each participant completed two tests: a theoretical test to assess TOE knowledge and a practical test using an online TOE simulator to investigate TOE skills. Among 716 young cardiologists from 81 countries, the mean theoretical test score was 56.8 ± 20.9 points, and the mean practical test score was 47.4 ± 7.2 points (/100 points max each), including 18.4 ± 8.7 points for the acquisition test score and 29.0 ± 6.7 points for the anatomy test score (/50 points max each). Acquisition test scores were higher for four-chamber (2.3 ± 1.5 points), two-chamber (2.2 ± 1.4 points) and three-chamber views (2.3 ± 1.4 points) than for other views (all P < 0.001). Prior participation to a TOE simulation-based training session, a higher number of TOE exams performed per week, and EACVI certification for TOE were independently associated with a higher global score (all P < 0.001).

Conclusion

Online evaluation of young cardiologists around the world showed a relatively low level of TOE skills and knowledge. Prior participation to a TOE simulation-based training session, a higher number of TOE exams performed per week, and the EACVI certification for TOE were independently associated with a higher global score.

Introduction

Learning transthoracic echocardiography (TTE) and transesophageal echocardiography (TOE) is a cornerstone of training for cardiology fellows in imaging, electrophysiology, and structural cardiology.^1,2 Although TTE is rapidly taught with dedicated training and practiced on patients during the cardiology fellowship, TOE remains too often neglected. The reference hands-on TOE teaching may be hampered by the availability of the teacher and equipment and by the procedure which is semi-invasive. Simulation may be a key educational tool to accurately assess TOE knowledge and skills among young cardiologists by allowing better targeting areas that need strengthening and thus accelerating the learning curve.³

Despite a growing interest in simulation-based training to assess the level of knowledge and skills of cardiologists, its accessibility remains very limited.⁴ In a recent international European Association of Cardiovascular Imaging (EACVI) survey performed with 172 young cardiologists from 43 countries, only 48% of the participants had already participated to a simulation session, whereas 91% considered this teaching approach as ‘necessary’ in cardiology.⁵

To the best of our knowledge, all studies that have evaluated the level of TOE skills of young cardiologists were based on: theoretical questions (multiple-choice questions) only, or practical simulator-based evaluation but with limited single-centre sample size.^6–8 One specific mission of the EACVI is ‘to promote excellence in clinical diagnosis, research, technical development, and education in cardiovascular imaging’. Therefore, the aim of this EACVI ‘SIMULATOR-online’ study was to assess the level of TOE knowledge and skills of young cardiologists across the world, using the first fully virtual simulation-based software.

Methods

Study population

The present study was conducted by the EACVI Heart Imagers of Tomorrow (HIT) Committee and EACVI Education Committee, following the guidelines of the EACVI Scientific Initiatives Committee.⁹ Using the EACVI survey network (www.escardio.org/eacvi) and the ESC social medias, all young cardiologists under 40 years eligible to participate of age across the world were invited to participate using advertising, direct email invitation and the national HIT Ambassadors network. The study was conducted during two periods from 9–12 December 2021 and 10–13 April 2022. All participants completed an anonymized online survey of 13 questions⁵ including demographics, training features and imaging competence, numbers and distribution of TTE and TOE exams already or currently performed, as well as thought about simulation (see Supplementary data online, Table S1). All participants gave their consent to participate to the study and agreed to provide honest answers about their practices. In accordance with the assessment of TOE knowledge and skills performed by several prior studies,^7,10–12 each participant completed two tests during the study: (i) a theoretical test using multiple-choice questions to assess TOE knowledge, and (ii) a practical test using an online TOE simulator to investigate the level of TOE skills.

Theoretical test

The theoretical test included 10 video-based multiple-choice questions online evaluating recognition of standard TOE views, normal anatomy and some pathological cases as already described.^10–12 This test was designed by the EACVI HIT Committee and reviewed by the EACVI Education and Scientific Initiatives Committees. All the 10 video-based multiple-choice questions were reported in Supplementary data online, Table S2. The participants were given 90 s for each question to choose the best answer out of a multiple choice of five propositions.¹² Each question was scored on 10 points (10 points if all the propositions were correct and 0 point if at least one error was observed), for a total of 100 points per test.

Practical test

Immediately after the theoretical test, all participants underwent the practical test. The participants were asked to show the 10 basic views required as standard in several previous reports,^7,8,10 using the TOE simulator-online software (3D Systems-Simbionix, Surgical Science, Littleton, USA, Figure 1). Each participant was given a maximum of 2 min for each view. These 10 required views are presented in Supplementary data online, Figure S1.¹⁰ The software timed the duration taken to complete the TOE examination. The practical test was automatically stopped by the software after a maximum of 20 min.

As already published,¹⁰ the practical test included two separate parts for each participant with a ‘TOE acquisition score’ on 50 points and a ‘TOE anatomy score’ on 50 points: (i) for the ‘TOE acquisition score’, a standardized assessment using the modified Ferrero grading scale was applied to data measured by the software to assess the quality of achievement of the views.^7,12,13 An example of the online TOE simulator is provided in Figure 1. For each view, 5 points scored for imaging angle and overall clarity of the view (poor quality = 0 point, average quality = 2 points, optimal quality = 5 points); (ii) for the ‘TOE anatomy score’, 5 points assessed the presence of all the pertinent anatomic structures in the view (−1 point per missing structure not shown, and zero point if no structure identified) (see Supplementary data online, Table S3). Therefore, each view was scored on 10 points, for a total of 100 points per practical test. Overall, a maximum global score of 200 points could be obtained for each participant (i.e. sum of the theorical and practical test).

Statistical analysis

Continuous data was reported as means ± standard deviation (SD) or mean for normally distributed data or as medians and interquartile range for non-normally distributed data, as assessed through graphical methods and the use of the Shapiro–Wilk test for normality. Categorical data was reported as counts and percentages. Between-groups comparisons were performed using Student’s t-test or Mann–Whitney test for continuous variables and using the χ² or Fisher’s exact test for categorical variables, as appropriate. Using violin-plots for assessing the relationship between the categorical variables and the different test scores, comparison tests were based on the Cochran–Armitage test for trends. We used multivariable linear regression analysis following a stepwise regression strategy with a P-value ≤0.20 on univariable screening to identify demographic data, independently associated with the global level in TOE for each participant. The correlation between each test score and the other variables was performed using Pearson’s correlation test for variables with a normal distribution, or Spearman’s correlation test for variables with a non-normal distribution. A two-tailed P-value <0.05 was be considered statistically significant. All data were analysed using the R software, version 4.1.1 (R Project for Statistical Computing, R Foundation, Vienna, Austria).

Results

Study population

Among the ∼4000 young cardiologists invited directly or through advertising, a total of 716 young cardiologists from 81 different countries around the world were included (see Supplementary data online, Table S4). The baseline characteristics of all participants are presented in Table 1. The mean age of the population was 31.9 ± 3.9 years, with as many men as women (women 50.6%), advanced in their curriculum with 46% of cardiologists, 37% of residents and 12% of fellows. Most participants have been performing TOE for less than a year (54%), with less than two TOE exams performed per week (69%) and no EACVI certification for TTE (83%) or for TOE (95%). While only 200 (28%) participants had already participated to an echocardiography simulation session, 90% considered this teaching approach as ‘very important’ or ‘crucial’, and 8% as ‘somewhat important’ in cardiology. Only 2.2% considered this teaching approach as ‘useless’ or ‘not so important’.

TOE knowledge and skills in young cardiologists

For all participants, the mean theoretical test score was 57 ± 21 points, and the mean practical test score was 47 ± 7 points (/100 points maximum each), including 18 ± 9 points for the acquisition test score and 29 ± 7 points for the anatomy test score (/50 points maximum each). The mean global test score was 104 ± 30 points (/200 points maximum) and the mean duration of TOE exam was 14 ± 4 min.

Regarding the theoretical test, the participants with more than 5 years practising TOE (60 ± 20 vs. 54 ± 21 points, P < 0.001), who performed more than five TOE exams per week (69 ± 17 vs. 55 ± 21 points, P < 0.001), and who held an EACVI certification for TOE (88 ± 7 vs. 55 ± 20 points, P < 0.001) displayed higher theoretical test score than others. Results were consistent regarding the practical test; participants with more than 5 years practising TOE (49 ± 8 vs. 46 ± 9 points, P < 0.001), who performed more than five TOE exams per week (70 ± 5 vs. 43 ± 7 points, P < 0.001), and who held an EACVI certification for TOE (75 ± 4 vs. 46 ± 7 points, P < 0.001) displayed higher practical test score than other participants (Figure 2). The correlation between the number of years practising TOE and each test score is depicted in Supplementary data online, Figure S2. In addition, the analysis assessing the impact of <1 vs. ≥1 year of TOE practice on all test scores is presented in Supplementary data online, Figure S3.

TOE acquisition and anatomy scores according to each of 10 views

The acquisition and anatomy test scores according to each of 10 TOE views are depicted in Figure 3. The maximum score for the acquisition test was 46/50 points. In the overall population, acquisition test scores were higher for four-, two-, and three-chamber views (respectively, 2.3 ± 1.5, 2.2 ± 1.4, and 2.3 ± 1.4 points) than for other views (all P < 0.001). At the opposite, descending aorta SAX/LAX, descending aorta SAX/LAX and bicaval views had the lowest acquisition test scores (respectively, 1.4 ± 1.3, 1.5 ± 1.4, 1.6 ± 1.5, 1.7 ± 1.2, and 1.7 ± 1.5 points; all P < 0.001). In addition, anatomy test scores were also higher for four-, two-, and three-chamber views (respectively, 3.6 ± 1.1, 3.6 ± 1.1, and 3.5 ± 1.1 points) than for other views (all P < 0.001).

Factors associated with a higher global score in TOE

Using multivariate linear regression analysis, the only three variables independently associated with a higher global test score were: having already participated in a TOE simulation-based training session [regression coefficient B (95% CI): 44 (29–91), P < 0.001], the number of TOE exams performed per week [B (95% CI): 188 (76–351), P < 0.001], and EACVI certification for TOE [B (95% CI): 276 (129–423), P < 0.001]. (Table 2).

Discussion

Based on the largest international online study assessing theoretical knowledge and practical skills in TOE among 716 young cardiologists from 81 countries around the world, our main findings are as follows: (i) the global TOE test score assessing both theoretical knowledge and practical skills was relatively low in young cardiologists (mean 104/200 points); (ii) the mean theoretical test score was better (57/100 points) than the mean practical test score (47/100 points); (iii) descending aorta, ascending aorta and bicaval views had the lowest acquisition test scores; (iv) the only three variables independently associated with a higher global score were: prior participation to a TOE simulation-based training session, a higher number of TOE exams performed per week, and EACVI certification for TOE.

Who may benefit from a TOE simulation-based programme?

Although recent studies suggested that simulation-based TOE teaching is displaying significant benefit over conventional methods in terms of improving TOE skills and accelerating learning,^6,8,14–16 there is currently no consensus or recommendation yet. Our results showed that prior participation to a TOE simulation-based training programme is associated with higher TOE knowledge and skills. These programmes may benefit to all cardiologists in training, and should probably be mandatory for fellows specializing in cardiovascular imaging to provide advanced TOE training. In addition, the place of TOE simulation-based is even more topical as a new specialty in structural heart disease interventional imaging is emerging.^1,17 Noteworthy, a TOE simulation-based training programme may benefit also to all people using cardiac ultrasound in daily practice (anaesthesiologists, intensive care specialists, sonographers…) and to any physician wishing to maintain, develop, or increase her/his skills and knowledge.

Tools available

Different TOE simulator manufacturers are available and all offer: (i) a normal TOE module, (ii) a module including different pathological cases, and (iii) a structural heart disease interventional module. Beyond the price, the implementation of a TOE simulation-based programme can be hampered by teachers’ availability. To overcome this issue, new virtual tools can be proposed to allow self-training of the student who repeats alone what she/he has learned during the simulation-based training sessions. In addition, several reports have showed that e-learning can assist simulation-based education in TOE and can complement traditional education techniques.¹⁸ This is an effective way for the student to maintain skills acquired during the courses over the long term. The present study suggest that an online TOE simulator is feasible in assessing knowledge and skills of young cardiologists. Further studies should explore how this virtual tool may be useful for training cardiologists and/or can be used as a complementary training tool, especially in centres with limited access to simulators.

Proposal for a TOE simulation-based training programme

A training programme will have to adapt to local resources, without awarding the hospital or health system credentials to perform the procedure in a clinical setting. Besides, the general requirements for training and competence in echocardiography according to EACVI recommendations still the same.² An assessment of knowledge and skills before the onset of the training is crucial to settle a dedicated and efficient teaching. The session should start by a quick overview of the educational goals. Short sessions in small groups should be offered (for example six students for a 2 h session, i.e. 20 min per student on a TOE simulator) to allow everyone to get sufficient handling time. The student should be informed that no simulator perfectly mimics the probe introduction yet. While one student manipulates the probe, other students are invited to watch and participate, continuing their learning with the teacher’s feedback. Afterward, a debriefing time is mandatory for relevant and personalized training. In addition, sessions will have to tailor to the level of expertise of the learners. For beginning students (Levels 1 and 2), sessions should focus on standard views and normal anatomical structures. For advanced students (Level 3), sessions should implement 3D views, pathological cases, and practical workshops on dedicated consoles. It might be interesting to time the real time of TOE completion (from probe introduction to probe removal) to verify the student improvement. The overall number of sessions might be adapted to the level and the progression of the students.

Impact of EACVI certification in TOE

In the current work, EACVI certification for TOE was independently associated with a higher global score. Indeed, participant with EACVI certification for TOE exhibited the highest mean practical test score compared with other participants. These findings emphasize the important impact of EACVI certification regarding TOE knowledge and skills for all young cardiologists (EACVI website: https://www.escardio.org/Education/Career-Development/Certification/Adult-Transoesophageal-Echo).¹⁹

Perspectives

In accordance with the EACVI recommendations for training, competence, and quality improvement in echocardiography,¹⁹ simulation-based training programme are the present and the future of TOE teaching. The relevance and implementation of this kind of programme is currently being tested by the ongoing randomized SIMULATOR study.¹⁰ Moreover, other solutions such as online simulators are emerging and will probably change the way of teaching and learning TOE in the years to come.¹⁸

Limitations

This study has some limitations. First, there is a risk of selection bias due to the fact that this survey was completed by volunteers with the aim of self-assessment to progress in terms of TOE knowledge, and young cardiologists with advanced TOE skills probably applied less for this survey. Second, it is important to mention that the number of TOEs performed worldwide during the COVID-19 pandemic had significantly decreased which may have led to a decrease in the overall knowledge and skill level of young cardiologists during the survey period. Third, the use of the simulator-online as TOE probe requires adaptation to best use the keyboard. Therefore, all participants were given a 5 min hands-on with the TOE simulator-online without any instruction to become familiar with the software before the beginning of the practical test. Fourth, the verification of the eligibility criteria linked to the age limit of 40 years was based on the declaration of the date of birth of the participants during their ESC registration, which therefore may have led to a risk of selection bias. As the number of participants with EACVI certification was relatively small (n = 72, 10%), the interpretation of the analyses using this variable should be taken with caution. Fifth, this study was not designed to assess the skills of TOE probe insertion, to manage the potential complications during an examination on real patients or to evaluate complex pathological cases; the aim of the current study was to evaluate the level on basic TOE skills and knowledge.

Conclusion

Online evaluation of young cardiologists around the world showed a relatively low level of TOE skills and knowledge. Prior participation to a TOE simulation-based training session, a higher number of TOE exams performed per week, and EACVI certification for TOE were independently associated with a higher global score.

Author contributions

T.P. and A.C. designed the trial. T.P. collected the data and performed statistical analysis. T.P., A.C., and B.C. analysed and interpreted the data. T.P. and A.C. wrote the manuscript. All the undersigning authors have substantially contributed to the paper. All authors reviewed the paper.

Supplementary data

Supplementary data are available at European Heart Journal - Cardiovascular Imaging online.

Acknowledgements

The production of this EACVI survey document is under the responsibility of the HIT Committee of the EACVI: B.M. (Chair), H.S. (co-chair), Rocío Blanco, Georg Spinka, Philippe Bertrand, Edin Begić, T.P., Marcelo Dantas Tavares De Melo, Fernanda Mello Erthal Cerbino, Gabriela Liberato De Sousa, Bernardo Baptista Da Cunha Lopes, Galina Zlatancheva, Gloria Adam, Pero Popeski, Christina Luong, Giulia Vinco, Dejan Dosen, Thrasos Konstantinides, Monika Kamasová, Reem Laymouna, Oliver Rosenbaum, Sanna Laurila, Elena Romero Dorta, Eirini Velegraki, Alexios Antonopoulos, Bálint Szilveszter, Golnaz Houshmand, Theo Murphy, Marco Malvezzi Caracciolo D’Aquino, Saima Mushtaq, Andrea Baggiano, Nobuyuki Kagiyama, Mohamad Mansour, Petras Navickas, Irina Kotlar, Kentaro Yamagata, Inessa Cojuhari, Han Naung Tun, Josephine Heindendael, Andreas Valeur, Karolina Kupczynska, Justyna Sokolska, Kevin Domingues, Joao Augusto, Diana Mihalcea, Beatrice-Simona Botezatu, Yosra Turkistani, Milica Stefanovic, Ondrej Benacka, Tomaz Podlesnikar, Marta Cvijič, Dong-Hyuk Cho, Aberto Cecconi, Rocio Eiros Bachiller, Christel Hermann Kamani, Dominic Benz, Panagiotis Antiochos, Anas Jano, Begum Uygur, Sanjay Bhandari, and Mykhaylo Kolesnyk. The authors thank Gilat Noiman and Ran Weiss from Surgical Science/Simbionix Simulators for the free access of TOE simulators-online and their technical support. The authors thank Oceane Marie, Audrey Esperou-Surrel, and Veronique Brassart from the ESC staff for their logistical support in setting up this EACVI study.

Funding

None.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.

References

Author notes