A well-balanced randomized controlled trial in 93 patients is more trustworthy than attempted propensity matching in 38 patients: comments on Schlachtenberger et al.

We were interested in the propensity score-matched analysis of head and neck cancer metastasized to the lung undertaken to ‘further clarify that PM prolonged survival’. The method for propensity score matching is not described [1]. The authors list the matching factors, but not the statistical method used for the matching; hence, it is difficult to comment on whether the matching was done appropriately. But from what is described there are several points in the paper, which suggests that the number of patients available was too small. For logistic regression, integral to propensity matching, the rule of thumb is that you need 10 patients per factor to be matched. Their Table 4 lists 14 factors which would require about 140 patients with the event for analysis while they have only 36. The text refers to Student’s t-test to compare the groups but it is a test of significant ‘difference’, not similarity. McNemar test is used for ‘paired’ data, for example to compare repeated measurements in the ‘same’ patient.

We were defeated in our own attempt to use propensity matching for colorectal cancer (CRC) lung metastases so we sincerely sympathize with the authors and regret the unavoidable statistical quibbling. One of us (Mišel Milošević) had a personal series of 73 well-documented CRC patients 2004–2012 and was granted access to data on 20 006 patients in the Registry of Cancer in Central Serbia. We made a determined attempt, but we did not trust the matches mainly because too few of the clinical factors could be matched in the registry. Siebenhuner et al. had more success. In the SEER database, they found 807 patients with lung only CRC metastases and 1323 with liver and lung with no survival benefit [2].

When there are no controlled studies a good propensity score matching analysis is a useful step. We appreciate Dr Schlachtenberger and colleagues making reference to the Pulmonary Metastasectomy in Colorectal Cancer (PuMiCC) study which in fact recruited well with 512 patients giving written informed consent and providing baseline data of trial quality [3]. These provided a prospective cohort of 391 patients. The clinical teams chose 263 for metastasectomy with 60% 5-year survival comparable with the best reported ‘real-world’ data and gave face validity to the PulMiCC cohort. A total of 128 whom they elected to not operate on had 22% survival, not the zero assumed in the STS Expert Consensus Document [4]. The chosen patients had better prognostic factors, often by a wide margin. The nested RCT 93 patients were excellently balanced for all prognostic factors—primary stage, number of metastases, carcinoembryonic antigen, interval since primary resection, liver involvement, age, performance status and lung function—and there was no difference in survival [3]. PulMiCC did not have statistical power to prove non-inferiority but any benefit is very much smaller than is widely believed [5]. Where there is an RCT, it is more reliable than attempts at matching, but when the numbers were large enough, the 2 methods were in accordance [2]. At least for CRC, a large benefit from lung metastasectomy is in doubt [6].

REFERENCES