Performance of a consensus-based algorithm for diagnosing anastomotic leak after minimally invasive esophagectomy for esophageal cancer

Abstract

Anastomotic leak (AL) is a common and severe complication after esophagectomy. This study aimed to assess the performance of a consensus-based algorithm for diagnosing AL after minimally invasive esophagectomy. This study used data of the ICAN trial, a multicenter randomized clinical trial comparing cervical and intrathoracic anastomosis, in which a predefined diagnostic algorithm was used to guide diagnosing AL. The algorithm identified patients suspected of AL based on clinical signs, blood C-reactive protein (cut-off value 200 mg/L), and/or drain amylase (cut-off value 200 IU/L). Suspicion of AL prompted evaluation with contrast swallow computed tomography and/or endoscopy to confirm AL. Primary outcome measure was algorithm performance in terms of sensitivity, specificity, and positive and negative predictive values (PPV, NPV), respectively. AL was defined according to the definition of the Esophagectomy Complications Consensus Group. 245 patients were included, and 125 (51%) patients were suspected of AL. The algorithm had a sensitivity of 62% (95% confidence interval [CI]: 46–75), a specificity of 97% (95% CI: 89–100), and a PPV and NPV of 94% (95% CI: 79–99) and 77% (95% CI: 66–86), respectively, on initial assessment. Repeated assessment in 19 patients with persisting suspicion of AL despite negative or inconclusive initial assessment had a sensitivity of 100% (95% CI: 77–100). The algorithm showed poor performance because the low sensitivity indicates the inability of the algorithm to confirm AL on initial assessment. Repeated assessment using the algorithm was needed to confirm remaining leaks.

INTRODUCTION

Anastomotic leak (AL) is a common and severe complication after esophagectomy. Recent literature reports leak rates up to 20%.^1,2 AL is associated with substantial postoperative morbidity, mortality, number of reinterventions, prolonged hospital care, and poor quality of life.^1–5 Diagnosing AL as early as possible is important to facilitate swift management and thereby may potentially improve patient outcomes.^6,7 Any delay in diagnosing AL negatively affects prognosis.⁸ However, diagnosing AL is challenging due to a diverse clinical presentation and lack of evidence to support a specific diagnostic strategy.^8,9 Hence, a gold standard for diagnosing AL is currently lacking.^7,9

For diagnosing AL, the most commonly used diagnostic modalities in current clinical practice are clinical presentation, laboratory investigations, imaging techniques and endoscopy. Often used laboratory investigations include leukocyte count, C-reactive protein (CRP) level, and drain amylase level.^10–13 Computed tomography scan, esophagogram and endoscopic evaluation are often applied (imaging) techniques for confirming AL.^9,11,13–15 However, it remains unclear which (order of) modalities should be used to diagnose AL. Different studies have suggested diagnostic algorithms, but performance of these algorithms has not been assessed.^11,16 Current clinical practice regarding diagnosing AL is mainly based on experience and an evidence-based algorithm for diagnosing AL might benefit patient outcomes.^7,9

The objective of this study was to assess the performance of a consensus-based algorithm for diagnosing AL after minimally invasive esophagectomy.

METHODS

Study design

This study used data of the ICAN trial; an open-label, multicenter randomized clinical superiority trial performed in 9 Dutch high-volume hospitals.^1,17 The ICAN trial compared the incidence of AL requiring reintervention after intrathoracic versus cervical anastomosis in patients undergoing total or hybrid transthoracic MIE with gastric tube reconstruction for esophageal cancer. The ICAN trial is registered in the Dutch trial register (NL4183 [NTR4333]). This study was approved as part of the approval of the ICAN trial by the institutional review board of the Radboud University Medical Center and all participating centers. All patients provided written informed consent. Current study was performed and reported in line with the Standards for Reporting of Diagnostic Accuracy Studies (STARD) guidelines.¹⁸

Algorithm

Prior to the start of the ICAN trial, participating hospitals agreed upon a consensus-based algorithm for diagnosing AL as standard postoperative care, irrespective of type of AL (Fig. 1). See Supplement 1 for in-depth information about the development of the algorithm. Types of AL correspond to the Esophagectomy Complications Consensus Group (ECCG) definition of types of AL: type I (AL requiring no change in therapy or treated medically or with dietary modification), type II (AL requiring endoscopic or radiologic intervention), and type III (AL requiring surgical intervention).¹⁹ The algorithm consisted of three steps. The first step focused on identifying patients suspected of AL. Patients were considered suspected of AL based on clinical sign(s) as interpreted by treating physician (e.g. fever, atrial fibrillation, respiratory complaints) on any postoperative day (POD), and/or a blood CRP level exceeding 200 mg/L, and/or a postoperative drain amylase level exceeding 200 IU/L. According to the research protocol, routine CRP and amylase laboratory investigations were performed on POD 2 to 5. The second and third steps of the algorithm focused on confirming AL in patients suspected of AL. During the second step, a swallow computed tomography (sCT) scan was performed using water soluble oral contrast according to local protocol. An endoscopy (i.e. the third step) was performed at the discretion of the treating physician in case of negative or inconclusive sCT scan result. In case AL was confirmed on sCT scan or endoscopy, treatment was performed according to local strategies. Physicians were allowed to deviate from the algorithm if deemed necessary for any reason.

Outcome measures

Primary outcome measure was algorithm performance in terms of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) regarding diagnosing AL. Secondary outcome measures included algorithm adherence and alternative complications than AL diagnosed by the algorithm. Algorithm adherence was defined as the rate at which the first two steps of the algorithm were performed (i.e. clinical tests and sCT scan). Since the choice to perform endoscopy after sCT scan was at the discretion of the treating physician, whether or not endoscopy was performed did not affect algorithm adherence.

Reference standard

Algorithm performance regarding diagnosing AL was determined by comparing the result of the algorithm (i.e. sCT scan and/or endoscopy result) with the reference standard. The reference standard was diagnosis of AL as determined by the data verification committee consisting of the coordinating investigator, principal investigator, and lead investigators of each participating hospital. The data verification committee evaluated each individual patient suspected of AL and determined whether AL was confirmed based on all available clinical data gathered during postoperative care, including CT-imaging, endoscopy findings and findings during reoperation or wound opening after suspicion of AL.^1,17 AL was defined as a full-thickness defect of the anastomosis confirmed by a computed tomography scan with intravenous and/or oral contrast, by an endoscopy, by drainage of ingested materials or saliva into the chest tube or at the cervical wound, during reintervention (including reoperation), or at autopsy.¹⁹ In line with the ICAN protocol, leakage or necrosis of the gastric conduit was recorded separately. If the location of a confirmed leak was uncertain, this was recorded as AL.^1,17

Data collection, storage, and validation

Data regarding suspicion of AL were prospectively collected by local clinicians in each participating hospital and recorded in online pseudonymized case report forms (www.castoredc.com). Repeated episodes of suspicion of AL were recorded if patients underwent a second sCT scan after initial inconclusive or negative outcome. Data validation was performed as previously published.¹

Statistical analysis

Frequencies and corresponding percentages were used for dichotomous data, while continuous data were presented as mean with standard deviation (SD) or median with interquartile range (IQR). Besides baseline characteristics, characteristics of patients stratified by leak suspicion were also determined.

Two-by-two tables were used to determine terms of algorithm performance. The algorithm regarding AL had three outcomes: positive, inconclusive, or negative. The inconclusive and negative outcomes were clustered to enable two-by-two comparison versus the positive outcomes based on the recommendations of previous research.²⁰

In case a patient was diagnosed with AL after repeated assessment using the algorithm, the first result of the algorithm was classified as ‘false-negative’ because AL was assumed present since it could probably have been present, but not diagnosed, during initial assessment.

Algorithm performance was determined based on initial assessment adhering to the algorithm, meaning patients were excluded from analyzing algorithm performance in case the algorithm was not adhered too. A subgroup analysis was performed for algorithm performance per anastomosis location and for the repeated assessments using the algorithm. Furthermore, a sensitivity analysis was performed to evaluate the impact of inconclusive algorithm results on algorithm performance. During this analysis, the inconclusive outcomes were excluded and algorithm performance was determined only based on comparing the positive and the negative outcomes.

Statistical analysis was performed with RStudio version 3.6.2. (RStudio) using package ‘epiR’.

RESULTS

Baseline characteristics

In the included 245 patients, cervical anastomosis was performed in 122 (49.8%) patients and intrathoracic anastomosis in 123 (50.2%) patients. All baseline characteristics are shown in Table 1.

Suspicion of AL

During the postoperative course, 125 (51%) patients were suspected of AL based on clinical signs or laboratory investigations (Fig. 2). Median POD of leak suspicion was day 5 [IQR: 3–7]. At leak suspicion, 68 patients (54.4%) only had clinical signs, 47 patients (37.6%) only had a CRP level above the threshold value, 5 patients (4%) had a CRP and drain amylase level above the threshold value and 5 patients (4%) only had a drain amylase level above the threshold value.

Algorithm adherence

The algorithm was adhered to in 110 out of 125 patients suspected of AL meaning algorithm adherence was 88% (Fig. 2). Of 15 patients in which the algorithm was not adhered to, 10 patients were diagnosed with AL by either exploration of the neck wound (n = 4), a clinical diagnosis (e.g. gastrointestinal fluid leaking from cervical wound, n = 4), a CT-angiography scan (n = 1) or autopsy (n = 1) instead of sCT scan. The other 5 patients were not diagnosed with AL after undergoing either CT without oral contrast (n = 4) or direct endoscopy (n = 1) instead of sCT scan.

Algorithm performance

The sCT scan result was inconclusive in 28 patients and negative in 56 patients. Further investigation with endoscopy was used in 19 patients (i.e. in 14 patients with an inconclusive and in 5 patients with a negative sCT scan result, respectively) and was positive in 5 patients, inconclusive in 5 patients and negative in 9 patients regarding AL (Table 2).

The algorithm was true-positive in 29 patients and true-negative in 49 patients. Median POD of leak confirmation by the algorithm was day 8 [IQR: 6–11]. Furthermore, the algorithm confirmed 67 alternative complications in 62 patients suspected of AL, but in 12 patients no cause for suspicion of AL was found. Alternative complications were pulmonary in 43 patients, infectious in 15 patients, gastrointestinal in 8 patients and cardiac in one patient (Table 2).

Based on initial assessment, the algorithm had a sensitivity of 62% (95% confidence interval [CI]: 46–75), a specificity of 97% (95% CI: 89–100), a positive predictive value of 94% (95% CI: 79–99) and a negative predictive value of 77% (95% CI: 66–86) for diagnosing AL (Table 3).

Eighteen patients with AL had a false-negative algorithm result (i.e. 11 patients negative and 7 patients inconclusive) on initial assessment. One patient (5.6%) with AL had an initial false-negative algorithm result due to a contradicting sCT scan interpretation between the radiologist and the treating physician who agreed to score the final interpretation as inconclusive, while the data verification committee determined AL was confirmed on initial assessment. Seventeen patients (94.4%) with an initial false-negative algorithm result were diagnosed with AL during repeated assessment using the algorithm.

Subgroup analysis

Repeated assessment using the algorithm was performed in 23 patients after initial inconclusive or negative algorithm outcome. The algorithm was adhered to in 19 patients. Median time between initial assessment and repeated assessment was 7 days [IQR: 5–11] (Fig. 2). The algorithm had a sensitivity of 100% (95% CI: 77–100) for patients with repeated assessment using the algorithm (Table 3).

In the subgroup of patients suspected of cervical AL, the algorithm had a sensitivity of 66% (95% CI: 47–81) and sensitivity was 53% (95% CI: 27–79) for patients suspected of intrathoracic AL.

The sensitivity analysis showed that the algorithm had a sensitivity of 72% (95% CI: 56–85) in case the inconclusive outcomes were excluded from analysis (Table 3).

DISCUSSION

This study evaluated the performance of a consensus-based algorithm for diagnosing any type of AL after MIE. Despite good algorithm adherence, the low sensitivity indicated that about 4 out of 10 patients with AL were not diagnosed on initial assessment irrespective of anastomosis location. Repeated assessment using the algorithm was needed to confirm remaining AL in case suspicion of AL persisted despite inconclusive or negative initial algorithm result.

The major strength is that this study is, to our knowledge, the first to assess the performance of an algorithm for diagnosing AL after esophagectomy. The algorithm was consensus-based, predefined, and comprised of modalities commonly used in current clinical practice in various centers. Thus, the algorithm reflects current diagnostic strategy regarding AL which explains good algorithm adherence. Therefore, current findings have important meaning for current clinical practice. Second, prospective, standardized registered and extensively reviewed data were used ensuring high data quality.

Several limitations should also be discussed. First, the ICAN trial was a randomized controlled trial and thus current analysis is not a true diagnostic study, but it is performed and reported in line with the STARD-guidelines. In addition, a generally accepted reference standard for diagnosing AL is currently lacking.^7,9 In our study, the verdict of the data verification committee was used as reference standard. This committee reviewed each individual patient suspected of AL using all available information and thus has provided a reliable reference standard. Consequently, our study provides accurate insight on the performance of the algorithm for diagnosing AL. Second, inconclusive results and repeated episodes of suspicion of AL are common in this study and handling these results poses a challenge in diagnostic research due to lack of a proper method to analyze these results, while ignoring them is counterproductive.^20,21 Current analysis handles the inconclusive results by clustering inconclusive and negative outcomes, and handles repeated episodes of suspicion of AL by assuming AL was already present, but not diagnosed, during initial assessment. These methods combined resulted in a substantial number of missed leaks on initial assessment. However, it remains uncertain whether these missed leaks were not detectable by sCT scan or endoscopy (i.e. AL too small to be visualized) or that there were in fact no leaks present to be detected. Both chosen methods prevent overestimation of the sensitivity and NPV, while underestimation cannot be ruled out. Third, generalizability of algorithm performance in patients who underwent open or transhiatal esophagectomy remains uncertain, because algorithm performance was only studied in patients who underwent MIE. Nonetheless, as the clinical presentation of AL after open or transhiatal esophagectomy may be similar to AL after MIE, comparable algorithm performance is expected. Besides, algorithm adherence was lower in patients with cervical anastomosis, as cervical AL was sometimes diagnosed clinically through apparent leakage from the wound or bedside exploration of the wound, without the need for CT imaging.

As reported in the ICAN trial, a total of 57 patients were diagnosed with AL, and only 3 patients were diagnosed with a type I leak meaning the vast majority (i.e. 94.7%) were type II and III leaks according to the ECCG classification.¹ Therefore, algorithm performance indicates the diagnostic performance of the algorithm predominantly regarding leaks requiring endoscopic, radiologic, or surgical intervention.

In comparison with previous studies, the median POD at which AL was confirmed by the algorithm in this study was day 8 which is in line with previous findings.^22–26 Although median POD of leak suspicion was day 5, the algorithm did not confirm AL earlier than other studies. The time between identifying patients suspected of AL and confirming AL indicates the diagnostic process with regard to AL is challenging. Furthermore, algorithm performance might be improved by reducing inconclusive algorithm results, because the sensitivity analysis showed the high number of inconclusive algorithm results negatively impacted algorithm performance substantially. First, adopting a scoring system for systematically assessing CT imaging might improve algorithm performance. Two studies found a superior diagnostic performance using a CT scoring system compared with subjective CT assessment. One study found a sensitivity of 80% and specificity of 84% for a 4-point prediction score for cervical AL.²⁴ Another study reported both sensitivity and specificity greater than 90% using a 5-point practical scoring system for intrathoracic AL.²⁷ Second, using endoscopy more often after inconclusive or negative sCT scan with persisting clinical suspicion might also increase algorithm performance. In our study, only half of the patients with an inconclusive sCT scan underwent endoscopy. Previous studies have shown that (routine) endoscopy had a high sensitivity. However, these studies are heterogeneous and limited by their single-institution and/or retrospective nature and/or biased by patient selection.^15,28–30 Based on the high sensitivity, one could argue to perform endoscopy as first diagnostic modality in patients suspected of AL. Nevertheless, the sCT scan remains preferred due to the ability to confirm various alternative complications in patients suspected of AL. Future studies may investigate routinely combining sCT scan and endoscopy to optimize diagnosing AL.⁸

From a clinical perspective, current findings show poor algorithm performance because the low sensitivity indicates the inability of the algorithm to confirm AL on initial assessment. The imaging modalities seemed to result in a substantial number of missed leaks, but it remains uncertain whether these were truly missed leaks. This uncertainty required persisting suspicion of AL and additional imaging to confirm the remaining leaks. Similar principle about missed leaks on initial assessment, to maintain suspicion of AL despite negative initial assessment and consequently to consider adjusting postoperative care, is reported.³¹ On the other hand, the algorithm seemed to be an useful and standardized tool to guide the diagnostic approach regarding various complications after MIE due to the ability to confirm a broad range of alternative complications in patients suspected of AL. Altogether, the clinical recommendation for treating physicians who persist in the suspicion of AL after inconclusive or negative initial assessment, is to initiate repeated assessment using the algorithm and to consider starting conservative treatment (e.g. broad-spectrum antibiotics).

In conclusion, despite good adherence, the algorithm for diagnosing any type of AL showed poor performance because the low sensitivity indicates the inability of the algorithm to confirm AL on initial assessment. Repeated assessment using the algorithm was needed to confirm remaining AL in case suspicion of AL persisted despite inconclusive or negative initial algorithm result. Treating physicians should remain cautious in case of inconclusive or negative algorithm result, initiate repeated assessment using the algorithm and to consider starting conservative treatment. Future research should aim to enhance interpretation of diagnostic (imaging) modalities to improve diagnosing AL after esophagectomy.

FUNDING

This study used data of the ICAN trial, which was funded by ZonMw. The funder had no role in the design, conduct, analysis or reporting of the trial and current study.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

DATA AVAILABILITY

Study data are not openly available, but authors are willing to share data on reasonable request at the corresponding author.

References