Non-technical error leading to patient fatalities in the Australian surgical population

Abstract

Introduction

Many surgical adverse events are due to errors in non-technical skills (NTS); consequently, improving NTS is a priority. However, evidence to guide NTS improvement activities is lacking. This study aimed to investigate the incidence and characteristics of non-technical errors linked to fatalities in a large, representative surgical-patient population to guide future NTS improvement.

Methods

All fatality cases with known or suspected adverse events reported to the Australian and New Zealand Audit of Surgical Mortality (ANZASM) between 2012 and 2019 were retrospectively assessed using a validated tool developed by the study authors. Outcomes included the incidence of non-technical errors linked to death (overall and by NTS domain), the identification of non-technical error predictors through multivariate analysis, and change in non-technical error incidence over time using statistical process control charts.

Results

Some 30 971 cases of surgical fatality were reported between 2012 and 2019, of which 3829 met the inclusion criteria. Due to insufficient information, 134 were excluded, leaving 3695 for analysis. Non-technical errors associated with patient death were identified in 63.7%. Of these, 58.4% had Decision-Making errors, 56.4% had Situational Awareness errors, 15.2% had Communication/Teamwork errors, and 5.44% had Leadership errors. Statistically significant predictors of Communication/Teamwork, Decision-Making, and Situational Awareness errors were identified. The incidence of overall non-technical errors decreased significantly between 2012 and 2019 and periods of significant decrease in Communication/Teamwork and Leadership errors were demonstrated. No significant decrease in Decision-Making or Situational Awareness errors were demonstrated.

Conclusion

The incidence of non-technical errors associated with surgical mortality rate is high. Future NTS improvement efforts should be targeted towards Decision-Making and Situational Awareness errors.

Introduction

Adverse events (AEs) occur in 23.6% of hospitalized patients¹. Despite technological advancements, novel surgical techniques, and the institution of surgical safety standards such as the WHO Surgical Safety Checklist, the percentage of AEs occurring among patients receiving surgical care remains unacceptably high at 30.4—50%^1–5. Given over 310 million surgical procedures are performed globally each year, this high rate of AEs represents a major problem^6,7.

Although technical competence is important, many AEs are caused by shortcomings in non-technical skills (NTS), the interpersonal and cognitive components of surgical professionalism^8–12. NTS shortcomings or ‘non-technical errors’ have been demonstrated to cause harm and death in multiple surgical specialties and across the entire surgical pathway including perioperative, intraoperative, and non-operative settings^8,13–16. NTS assessment and improvement has therefore become a priority for surgical training organizations across the world^17–19.

However, the impact of non-technical errors on patient safety, and the circumstances in which they occur, is poorly understood. Previous studies investigating AEs caused by non-technical errors included small cohorts, non-generalizable to the broader surgical patient population, or assessed for errors using inconsistent and non-comprehensive NTS domains^{9,10,13–16,20,21}. Therefore, the incidence and characteristics of non-technical errors in a large, generalizable, surgical patient cohort is unknown.

The purpose of this study was to provide a comprehensive analysis of non-technical errors associated with patient death in a large, generalizable, surgical patient cohort using the System for Identification and Categorization of Non-technical Errors in Surgical Settings (SICNESS)²². By doing so, future NTS improvement activities can be better directed and AEs may be reduced.

Specific aims:

1. Investigate the incidence of non-technical errors linked to surgical patient death over an eight-year period, in all surgical specialties in Australia.

2. Categorize identified non-technical errors into one of four NTS domains—Communication/Teamwork, Decision-Making, Situational Awareness, and Leadership.

3. Investigate predictive factors of non-technical error occurrence.

4. Investigate change in non-technical error incidence over time.

Methods

An eight-year retrospective audit of prospectively collected surgical fatality cases from the Australia and New Zealand Audit of Surgical Mortality (ANZASM) was undertaken. Ethical approval was granted by the Royal Australasian College of Surgeons (RACS) Ethics committee. This study was reported in accordance with the STROBE guidelines²³.

ANZASM is a mandatory, peer-reviewed, national audit overseen by RACS with 100% public hospital and 97% surgeon participation²⁴. Details of the ANZASM process are described elsewhere^24,25. In brief, all surgical patient fatalities in Australia are reported to ANZASM for peer-reviewed assessment, excluding New South Wales (NSW). NSW surgical fatalities are reported to the Collaborating Hospitals Audit of Surgical Mortality (CHASM) and independently managed by the Clinical Excellence Commission of NSW²⁵. Therefore, cases from NSW were unavailable for this study. ANZASM includes patients who died under the care of a surgical team, regardless of whether they underwent surgical intervention. For every death, details pertaining to patient management are reported by the involved surgeon. This report is de-identified and sent for external peer review by an independent consultant surgeon to assess the care provided and identify any clinical management issues (CMI). Cases are flagged with a CMI if there is legitimate concern that a component of patient management may have contributed to patient death. The severity of concern is then graded as either an area for consideration, an area for concern, or an AE²⁵. If a conclusion cannot be reached by the first-line reviewer, the case is escalated to another consultant surgeon for further assessment.

All ANZASM surgical fatality cases occurring over an eight-year period (January 2012–December 2019) flagged with an area of concern or AEs, across all surgical specialties in Australia, were included for analysis. The highest level of external peer review available for each case was used. No further exclusions were applied.

Cases were retrospectively assessed using the SICNESS, a tool developed by the study authors²². This tool was designed for retrospective assessment of surgical patient information to identify and categorize non-technical errors leading to patient death. Each case was assessed by two independent reviewers, to identify if a non-technical error had occurred (Aim 1), and to which of the four NTS domains the identified errors belong—Communication/Teamwork, Decision-Making, Situational Awareness, or Leadership (Aim 2). Disagreements between reviewers for Aim 1 were independently reviewed and resolved by a third reviewer (J.B.N.), a senior consultant orthopaedic surgeon with extensive experience in teaching and assessing non-technical skills at a national level. Disagreements between reviewers for Aim 2 were resolved by inter-reviewer discussion guided by the SICNESS tool user manual and validated exemplars, until consensus was achieved.

The review team included five individuals: a researcher with expertise in surgical NTS assessment, education, and research (J.D.E.), two general surgery education and training program trainees, both with substantial clinical experience and NTS expertise gained through RACS-approved courses, and previous NTS-specific research and assessment activities (V.K., M.B.H.), and two final-year medical students (O.L., K.H.). Prior to assessment, reviewers underwent training in surgical NTS and an education/standardization phase overseen by the first author (J.D.E.) and the senior author (G.J.M.), a Professor of Surgery and Consultant general surgeon with extensive experience in NTS research and assessment. For the standardization phase, each reviewer assessed 185 surgical fatality cases, approximately 5% of the entire study cohort. An initial 90 cases were assessed, compared, and discussed with the senior author. Thereafter, the remaining 95 cases were independently assessed. Greater than 80% agreement was achieved between reviewers demonstrating acceptable internal validity.

The incidence of identified non-technical errors, and their specific domains, were analysed using descriptive statistics reported as percentage of all included cases. Multivariable binary logistic regression was performed to investigate the relationship between non-technical error occurrence, overall and by NTS domain and several predictors: admission type (elective versus emergency), patient age, patient sex (male versus female), hospital status (private versus public), and patient status (private versus public). Change in non-technical error incidence over time was assessed using statistical process control (SPC) charts. Odds ratios and 95% confidence intervals were reported where appropriate. P less than or equal to 0.05 was considered statistically significant for all analyses. The statistical software used was SAS On Demand for Academics (SAS Institute Inc.2024 version 9.4: Cary, NC, USA).

Results

Between 2012 and 2019, 30,971 surgical fatality cases were reported to ANZASM, of which 3829 were flagged with an area of concern or an AE. This represents 12.4% of all surgical fatalities in Australia (excluding NSW) within the study period. Some cases (134/3829, 3.25%) were excluded due to insufficient information to apply the SICNESS tool, leaving 3695 cases for full analysis.

Non-technical errors linked to patient death were identified in 2354/3695 cases (63.7%). Of the cases with non-technical errors, 1375/2354 (58.4%) had Decision-Making errors, 1328/2354 (56.4%) Situational Awareness errors, 357/2354 (15.2%) Communication/Teamwork errors, and 128/2354 (5.4%) Leadership errors.

Of cases with a non-technical error, 1648/2354 (70.1%) had errors from only one NTS domain, 589/2354 (25%) had errors in two NTS domains, 106/2354 (4.5%) had errors in three NTS domains, and 11/2354 (0.5%) had errors in all four NTS domains.

Multivariable analysis

A statistically significant association was found between Communication/Teamwork errors and Hospital status, with patients admitted to private hospital half as likely to suffer a Communication/Teamwork error than those admitted to public hospitals (P = 0.04). There was a statistically significant association between Decision-Making non-technical errors and both admission type (elective versus emergency), P = 0.01, and patient age, P = 0.02. Elective patients were 1.32 times more likely to have a Decision-Making error and with every 10 years’ increase in age patients were 1.07 times more likely to have a Decision-Making error. A statistically significant association between Situational Awareness errors and age (P = 0.01) was demonstrated. For every 10 years of age there was a decrease in the likelihood of a Situational Awareness error (Table 1).

Non-technical error incidence over time

Several periods of special cause variation and an overall improvement were demonstrated for non-technical errors overall, meeting the definition of special cause improvement (Fig. 1). For Communication/Teamwork errors and Leadership errors, SPC charts demonstrated periods of special cause variation but did not show special cause improvement overall (Figs 2a,b). No overall improvement or periods of special cause variation were demonstrated for Decision-Making or Situational Awareness errors over time (Fig. 2c,d).

Discussion

This is the first study to provide clear, quantifiable evidence of the significant role non-technical errors play in surgical AEs and death.

AEs lead to preventable morbidity and mortality rates, and contribute significantly to hospital expenditure and resource use^26,27. Previous studies have used diagnostic or clinical measures to determine the incidence, cause, and mechanism of death caused by AEs such as circulatory failure, respiratory failure, or sepsis^28–30. However, few studies have taken a further step to explore the causes of the AEs themselves.

Studies have repeatedly demonstrated that NTS and human factors play a significant role^{9,10,13–16,31}. These studies are limited by the inclusion of small cohorts and heterogeneous non-technical error assessment methodology, making meaningful comparison difficult.

The present study is the first to report an overall incidence of surgical fatality attributable to non-technical errors using validated NTS domains. Although no formal consensus on NTS domains exists, the domains included in the SICNESS tool are representative of the full spectrum of surgical NTS validated by Modified Delphi Process, and mirror the most widely used NTS assessment tool^12,22. Study data include a wide range of demographic variables, across 15 surgical specialties, and AEs relating to perioperative, intraoperative, and non-operative environments, making these results generalizable to the broader surgical population and therefore a sound starting point to guide future NTS improvement activities^25,32,33.

This study is also the first to identify predictors of non-technical error occurrence.

Age was a significant predictor for both Decision-Making and Situational Awareness errors, yet with opposite effects. For every 10-year increase in patient age, the likelihood of Decision-Making errors increased whereas the likelihood of Situational Awareness errors decreased. Decision-Making around operative timing and choices is always difficult³⁴. As well as the presenting surgical pathology, elderly patients tend to bring reduced age-related physiological reserve and higher co-morbidity burden. Older patients are also more likely to be admitted under emergency conditions, necessitating prompt and decisive management actions. Combination of these factors often results in surgeons having to decide between palliating the patient or offering high-risk operative management with low chances of success^33,34. In these conditions, decision-making is as much art as it is science and, inevitably, some errors will occur³⁵. Decision-Making errors extend beyond operative choices and occur at all stages of the surgical patient journey. Although some errors are inevitable, many are preventable. Conversely, older age was protective against Situational Awareness errors. Previous studies have demonstrated that age is an independent factor for increased ordering of laboratory and investigative testing, suggesting that doctors are more diligent in their efforts to understand and assess the conditions of elderly patients, perhaps owing to the difficulties and high-risk nature of definitive decision-making previously described^36,37.

Communication/Teamwork errors were half as likely to occur in private hospitals compared to public hospitals. This may be explained by the fact that in private hospitals, patients are primarily under the care of an individual surgeon rather than a larger team. Communication in this setting often occurs at a senior level, involving fewer personnel, and therefore, the risk of a communication error is reduced. Conversely, patients in public hospitals are often more unwell and medically/surgically complex, necessitating the involvement of multiple teams and individuals, increasing the risk of communication errors.

Although hospital type was a significant predictor of non-technical errors, patient status (private versus public) was not. Although most public patients are treated in public hospitals, and private patients treated in private hospitals, there is a relatively large cohort of private patients who undergo treatment in public hospitals. This finding suggests that the occurrence of non-technical errors is influenced by components intrinsic to the hospital (such as infrastructure, staffing, and culture) rather than patient demographic factors.

Patients admitted electively were more likely to incur a Decision-Making error than those admitted emergently. Elective admissions are often lower acuity and may be expected to follow a ‘textbook’ management plan. Consequently, deviations from the expected post-operative course may be overlooked. Due to the routine nature of many elective admissions, it is conceivable that less senior input is required, and care may be entrusted to more junior members of the team, increasing the likelihood of Decision-Making errors. Conversely, decision-making in high-acuity, emergent situations is complex and often there is no ‘correct decision’. In these circumstances, a decision that results in death, although unfortunate, may not be considered an ‘error’ due to a lack of reasonable alternatives. Decisions that lead to death in routine, elective admissions may be more readily identified with clear ‘correct’ options available.

It is not clear why these factors are significantly associated with the occurrence of non-technical errors and causational explanation is beyond the scope of this study. Surgical care is complex and non-technical error occurrence likely multifactorial. The predictors assessed in this study were not exhaustive and further investigation is needed to understand the impact of other variables such as surgical specialty, rurality, and specific diagnoses/management protocols on non-technical error occurrence.

The incidence of non-technical errors overall significantly decreased between 2012 and 2019 and periods of significant improvement were demonstrated for Communication/Teamwork errors and Leadership errors. This is the first study to demonstrate any meaningful reduction in non-technical errors using longitudinal data. These results do not describe a total decrease in surgical mortality rate as only a subset of fatalities from the study period were included. However, as the non-technical errors identified in this study were explicitly linked to death, this reduction does mean there were significant fewer deaths caused by NTS failures.

The decrease in non-technical errors may be attributable to increased awareness of NTS through an expanding body of literature or the inclusion of NTS as core competencies of RACS¹⁷. More likely, the ANZASM process is responsible for the significant reduction, being the only mandated and widespread national improvement activity occurring over the study period^24,38. ANZASM includes specific questions about NTS at each stage of peer review, explicitly enabling the identification of non-technical errors and the provision of NTS-related feedback. The findings of this study justify the continuation of ANZASM in Australia and should encourage the initiation of similar processes in other countries/jurisdictions.

Despite the significant decrease in non-technical errors observed during the study period, the overall incidence is still unacceptably high. In 2019, the most recent year assessed, non-technical errors linked to patient death were identified in more than 50% of cases. Furthermore, over the eight-year period, no significant changes in the two most common NTS domain error types, Decision-Making or Situational Awareness, were demonstrated. This is significant and should serve as a call to action for surgical training organizations and the greater surgical community.

Previous studies have demonstrated that surgeon and surgical team NTS can be improved overall, and by NTS domain, yet this has not translated into improvements in patient outcomes^39–42. This is likely because previous improvement activities have been surgeon centric, directed at outcomes such as self-confidence, technical competence, reduced anxiety, or adherence to standardized protocols, rather than the reduction of patient harm^43–48.

Reducing patient harm must be the priority for future NTS improvement activities. The results of this study provide evidence to direct future efforts. First, Decision-Making and Situational Awareness errors have been identified as clear NTS priorities for NTS training.

Second, this study highlights the utility of retrospective NTS assessment. The SICNESS can be applied to complex surgical data, enabling identification of discrete non-technical errors in a comprehensive and standardized manner. The SICNESS can be implemented at all levels of quality assessment and improvement in surgery. At a local level, it could be introduced as an adjunct to clinical audit processes such as morbidity and mortality meetings, making NTS assessment universally feasible. In doing so, common and severe non-technical errors can be identified, timely and specific learning opportunities can be provided, and patient harm can be prevented.

This study is not without limitations. The study cohort is representative of a generalizable patient population, in terms of demographics, surgical specialties, and clinical contexts. However, only fatality cases were included; therefore, the results only reflect a proportion of all surgical patients. Patients who suffer AEs leading to morbidity without death have not been represented in this study and further research is needed.

Another limitation is the lack of fatality data from NSW. NSW is the largest state in Australia comprising one-third of the population. It is possible that these missing data may impact the study results, but due to the oversight of RACS, the national surgical training organization, surgical standards and practice are sufficiently similar, making this unlikely.

Finally, these results are derived from analysis of written documentation. The ability to meaningfully assess historical documentation to identify event occurrence has been previously scrutinized; the alternative method is interviewing individuals involved in the events. Interview methodology is financially and logistically prohibitive and prone to a multitude of recall biases, rendering this an equally if not more unreliable method²². Interpretation bias was minimized by using the SICNESS, a reliable and valid tool, with transparent methodology and a comprehensive user manual enabling readers to understand clearly how these results were reached.

Funding

J.D.E. and E.C.T. received The University of Adelaide Research Training Program Scholarship and the Basil Hetzel Institute Higher Degree top up scholarship. M.B.H. received the University of Adelaide Research Training Program Scholarship, the South Australian Hospital Research Foundation Higher Degree Scholarship, and the Basil Hetzel Institute Higher Degree top up scholarship. No industry or other funding was received for this work. The data used for this study were collected, collated, and paid for by the Royal Australasian College of Surgeons.

Acknowledgements

The authors thank Suzanne Edwards, senior biostatistician, for their involvement in the statistical analysis of this study.

The authors would also like to thank the Royal Australasian College of Surgeons and ANZASM team for approving this study and providing the data.

Author contributions

Jesse D. Ey (Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing—original draft, Writing—review & editing), Victoria Kollias (Conceptualization, Data curation, Formal analysis, Investigation, Validation, Writing—review & editing), Octavia Lee (Data curation, Formal analysis, Investigation, Writing—review & editing), Kelly Hou (Formal analysis, Investigation, Writing—review & editing), Matheesha B. Herath (Conceptualization, Formal analysis, Formal analysis, Investigation, Investigation, Writing—review & editing, Writing—review & editing), John B. North (Formal analysis, Supervision, Validation, Writing—review & editing), Ellie C. Treloar (Conceptualization, Formal analysis, Writing—review & editing), Martin H. Bruening (Conceptualization, Project administration, Supervision, Writing—review & editing), Adam J. Wells (Conceptualization, Project administration, Supervision, Writing—review & editing), and Guy J. Maddern (Supervision, Conceptualization, Project administration, Writing—review & editing).

Disclosures

No conflicts of interest to disclose.

Data availability

Due to the sensitive nature of this research the data is not available for sharing.

This paper is not based on previous communication to a society or meeting.

References