Reply to Nezic

We thank Dusko Nezic for his interest in our article [1, 2]. With regard to the first issue concerning operative mortality, we are happy to report that all patients classified as operative mortality were not discharged from the (base) hospital prior to death, prior to 30 days. Therefore these patients fulfil the criteria for operative mortality for both scores.

The second concern that performing EuroSCORE II (ESII) validation on patients operated years before initiation of ESII may be potentially misleading is challenging and complex. One would assume that any scoring system going forward in time would ‘drift’ from acceptable levels of model calibration due to dynamic changes in patient characteristics, case-mix and baseline risk. Using a scoring system to validate operative mortality of patients undergoing surgery years before the inception of a score might be inaccurate due to similar ‘drift’ but in a reverse direction. Others have expressed this concern: Hickey et al. [3] in a 2013 Letter to the Editor questioned the usefulness of retrospective performance of ESII in a study by Chalmers et al. [4], in which 5576 subjects between January 2006 and March 2010 were evaluated retrospectively by the 2012 ESII. The authors’ response [5] acknowledged that time bias might likely exist in single centre validation studies but that a seasonal bias in the ESII may be more serious. In addition, we also were concerned with a potential time bias and for our study looked at the change in both the calibration and mortality differences over time; there were none. (Please see the ‘Calibration’ section of the ‘Results’ [2].)

Scoring system analyses are difficult to produce, analyse and validate in a timely fashion because of the huge numbers necessarily involved. Indeed, there was a 2 year delay from performance of the surgeries for the analysis of ESII (May–July 2010) to the publication of the ESII manuscript (2012). The above mentioned authors [5] pointed out that validation using the same data collection time frame as the ESII would need a large national dataset, and then might be 3–4 years out of date by the time the data would be ready.

There is no data yet to support a ‘time bias’ hypothesis of the examination of the performance of ESII on retrospective samples. There is actually evidence of the opposite: in a very comprehensive meta-analysis (67 articles) of the performance of ESII, Siregar et al. [6] demonstrated that contrary to speculation that changes in patient characteristics and technological improvements accounted for poor calibration in the past, they found an opposite trend, that is, overestimation was present from the beginning and deterioration in the score over time could not be demonstrated despite the ‘16 years that had passed’. They also agreed with our conclusion that the logistic ES overestimates mortality.

It is our opinion that the ESII model has had many more changes than changes in patients, baseline risk and case-mix. With improved sophistication of statistical methodology more variables can be considered.

We thank Nezik for this stimulation of thought and useful discussion.

REFERENCES