Proficiency Testing to Assess Technical Performance for CTC-Processing and Detection Methods in CANCER-ID

Abstract

Background

Multiple technologies are available for detection of circulating tumor cells (CTCs), but standards to evaluate their technical performance are still lacking. This limits the applicability of CTC analysis in clinic routine. Therefore, in the context of the CANCER-ID consortium, we established a platform to assess technical validity of CTC detection methods in a European multi-center setting using non-small cell lung cancer (NSCLC) as a model.

Methods

We characterized multiple NSCLC cell lines to define cellular models distinct in their phenotype and molecular characteristics. Standardized tumor-cell-bearing blood samples were prepared at a central laboratory and sent to multiple European laboratories for processing according to standard operating procedures. The data were submitted via an online tool and centrally evaluated. Five CTC-enrichment technologies were tested.

Results

We could identify 2 cytokeratin expressing cell lines with distinct levels of EpCAM expression: NCI-H441 (EpCAM^high, CK^pos) and NCI-H1563 (EpCAM^low, CK^pos). Both spiked tumor cell lines were detected by all technologies except for the CellSearch system that failed to enrich EpCAM^low NCI-H1563 cells. Mean recovery rates ranged between 49% and 75% for NCI-H411 and 32% and 76% for NCI-H1563 and significant differences were observed between the tested methods.

Conclusions

This multi-national proficiency testing of CTC-enrichment technologies has importance in the establishment of guidelines for clinically applicable (pre)analytical workflows and the definition of minimal performance qualification requirements prior to clinical validation of technologies. It will remain in operation beyond the funding period of CANCER-ID in the context of the European Liquid Biopsy Society (ELBS).

Introduction

Circulating tumor cells (CTCs) are cancer cells that have entered the blood stream and can become detectable in the peripheral blood. CTCs have become of high interest since they can provide direct access to systemic cancer hallmarks with the potential to develop superior assays for detection, analysis, and treating systemic cancer (1–4). When compared to other circulating biomarkers, the clear advantage of CTCs is that they can provide cancer-related information on the DNA, RNA, and protein levels, which might be used for more rational treatment decisions.

Since CTC concentration in blood is extremely low and cancer specific markers are lacking, their enrichment and detection remains very challenging (5). In addressing these challenges, several platforms have been established to enrich and detect CTCs in blood samples. According to their underlying principle, current CTC-enrichment methods can be divided into 2 groups: one targets biological properties of CTCs (marker dependent) and the other their biophysical characteristics (marker independent) (6, 7). The prevailing strategy for the first group is immuno-magnetic enrichment via conjugated antibodies directed against EpCAM, a membrane-protein widely expressed in different cancer types (8). This first strategy avoids the loss of deformable CTCs with small sizes. The second group utilizes differences in size, deformability, or electric properties to capture CTCs. This could be advantageous if cells do not express typical epithelial markers such as EpCAM [e.g., due to epithelial-to-mesenchymal transition (EMT)] (9). EpCAM-based enrichment is also employed in the CellSearch^® system. This system is FDA-cleared for CTC analysis in metastatic breast, prostate, and colorectal cancers (10, 11). However, the prognostic significance of CTCs detected by CellSearch has been demonstrated for several cancer entities (12), including non-small cell lung cancer (NSCLC) (13). Since the introduction of the CellSearch^® system around 15 years ago, more than 40 CTC-detection systems have become commercially available. In view of the promises for therapy prediction in personalized medicine, the global forecasts for the CTC market have been very optimistic with compound annual growth rates of up to 23% to attain an expected global market value of around 28 billion USD by the end of 2023 (14). In light of these developments, it is surprising that for almost all available CTC-technologies (with the exception of the CellSearch^® system) large multicenter trials to show clinical validity have been missing, a fundamental requirement for more complex trials testing predictive value or even their routine clinical use. An obvious reason for this is the associated costs, which are difficult to cover for small and medium sized enterprises (SMEs) or academic groups, from which most technologies emerged.

However, an essential step for each CTC-detection method on the way to demonstrate clinical utility is to establish first its technical validity. This is already a complex, time consuming, and expensive task that appears challenging for most technology developers. This dilemma and the wish to support current and future CTC-technology developers triggered the idea to establish a sustainable proficiency testing platform to benchmark technical performance for CTC-detection methods and to allow for independent comparison of different technologies. The proficiency testing platform was developed within the frame of the Innovative Medicines Initiative (IMI) consortium CANCER-ID (15), which aims to test standard operating procedures (SOPs) for preanalytical sample handling and detection of CTCs in NSCLC as a blood-based biomarker. The increasing number of treatment options for patients with NSCLC has created a need for biomarkers to stratify patients and/or to monitor patient’s response. In this context, CTCs could be a valuable source of tumor material fulfilling this need; however, detection of NSCLC CTCs remains a major challenge due to their rarity and their potential phenotypical heterogeneity. Here, we present the first results of this proficiency testing platform for CTC enrichment/isolation/detection technologies. As quality control materials, we used NSCLC cell lines spiked blood.

Materials and Methods

Preparation and Shipping of Spiked Samples

Blood collection and preparation of spiked samples were done at Integrated Biobank of Luxembourg (IBBL). On the day of spiking, blood was collected from 1 healthy donor under informed consent “CNER: 201107/02” version 1.3 and following amendments (approved by the local Luxembourg ethic committee) directly into the blood collection tubes (BCTs) indicated in the SOPs of the different technology providers: CellSearch^® and VyCAP Microsieves required CellSave Preservative Tubes (Menarini), Siemens and Parsortix^® used CTC TransFix EDTA tubes (Cytomark), and RareCyte used AccuCyte^® BCT (RareCyte). NCI-H441 and NCI-H1563 cell lines in culture (Supplemental Methods) were harvested following standard treatment with trypsin, and quantified and evaluated for their viability and cell size using the Cell Counter CASY (OLS). Subsequently cells were stabilized with the fixation solution from the BCT required by the respective SOP, diluted to 15000 cells/mL and stained with Hoechst 33342 nuclear marker (Thermo Scientific). For the RareCyte spike in tests, cells were stained with 0.5 nmol/L Syto-83 (Thermo Fisher Scientific) because of possible interference of Hoechst with the downstream analysis. Two to 3 drops of 2 µL of stained cell suspension were placed unto a 1% BSA/PBS precoated microscope slide. Stained nuclei were counted by fluorescence microscopy, and if in the range of 50 to 100 cells in total, flushed into the collected blood with 200 µL PBS. On the same day, the spiked blood samples were shipped to the participants’ sites by overnight shipment via courier. Shipping conditions were according to the provided SOPs: the CellSave ensures sample stability for 4 days at a temperature range of 15°–30 °C; the TransFix ensures sample stability for 5 days at a temperature range of 18°–25 °C and required special room temperature isothermal packaging. The AccuCyte^® BCTs have a maximum processing delay of 3 days and require a transport temperature range of 20°–25 °C. This procedure was repeated twice using blood from a different donor each time, with 1-week intervals, resulting in 3 enumerations with blood from 3 different donors, per platform per cell line per site. All the participant sites were informed in advance of the shipping dates.

Enrichment of CTCs and Report of Results

Five methods for CTC-enrichment were tested: the CellSearch^® system (Menarini Silicon Biosystems); VyCAP Microsieves (VyCAP); Siemens filtration unit (Siemens; prototype is not for sale); Parsortix^® (ANGLE); and RareCyte platform (RareCyte) (Table 1), using consensus SOPs (Supplemental Methods). For all technologies, detection of spiked cells after enrichment was based on immunofluorescence staining and fluorescent microscope imaging (Supplemental Methods). Results were reported using an online questionnaire made available to the sites in the secured web-based reporting tool within the Biospecimen Proficiency Testing Programme of Integrated BioBank of Luxembourg (IBBL) (16) (Supplemental Methods, Supplemental Fig. 1).

Statistical Analyses

Statistical analyses were performed as described in the Supplemental Methods.

Results

Rationale for the Selection of NCI-H441 and NCI-H1563 Lung Cancer Cell Lines for Ring-Trials

For our testing platform for CTC-detection methods, we aimed to provide 2 lung cancer cell lines with strong differences in the expression of EpCAM, frequently used as cell surface antigen for label-dependent enrichment of CTCs (7, 17). In particular, we aimed to identify cell lines fulfilling 3 criteria: 1) All cell lines should express cytokeratin (CK), since this has become the standard marker for CTC identification by immunostaining; 2) Cell lines should have a differential expression of EpCAM to test marker-dependent and -independent platforms; 3) Cell line cells should be different in size to test size-dependent methods. Of the 21 NSCLC cell lines available among the CANCER-ID partners, we selected 9 for further analysis based on their EpCAM and CK RNA expression levels (Supplemental Fig. 2). As expected, EpCAM expression correlated positively with the expression of E-cadherin (CDH1) and other epithelial markers among the different NSCLC lines, but negatively with the mesenchymal markers vimentin (VIM) and N-cadherin (CDH2) (Supplemental Fig. 2C). Using flow cytometry, we quantified then the exact number of EpCAM (range: 4.3 x 10³—1.1 x 10⁶ antibodies bound per cell [ABC]) and CK (range: 1.6 x 10⁴—1.1 x 10⁵ ABC) epitopes per cell for each cancer cell line (Supplemental Figs. 3 and 4). Next, we determined the cell size, which did not vary substantially between the cancer cell lines (range: 14.7 - 18.2 µm) (Supplemental Fig. 3). Based on these results, we chose one EpCAM^high cell line (NCI-H411) and one EpCAM^low cell line (HCI-H1563) (Fig. 1A–D). Notably, each of the 2 cancer cell lines harbored a unique mutation pattern in KRAS, TP53, and EGFR genes (Supplemental Fig. 2D), and high intra-line genomic similarity (Fig. 1E).

Recovery of Spiked Cells Using Different Technologies

To test the 5 CTC-enrichment methods, a total of 98 samples were prepared at the central laboratory (IBBL, Luxemburg) and shipped to 9 participating sites in 6 countries (Germany, France, Belgium, Netherlands, Switzerland, Italy) according to the recommendations of the manufacturers of the preservative tubes (Fig. 2A and B). Siemens and VyCAP platforms were available at 4 participating sites, while Parsortix, CellSearch, and Rarecyte were tested in 3 sites (Table 2). The number of cells spiked per sample ranged from 51 to 99 (mean: 77.5; median: 78) and it did not differ significantly between the samples to be analyzed with the different technologies. The mean viability over all the spike-in experiments for NCI-H441 was 95.1% (range: 91.8%—98.4%) and for NCI-H1563 was 96.6% (range: 93.0%—98.1%), and no significant variation in cellular size was observed during the experimental period.

In the present work, a majority of samples arrived at the participating sites on the next working day (within 24 h after samples preparation). The exceptions were 2 of the 27 shipments where there was a 1 day delay in delivery; as this was still within the recommended time limits of the tube manufacturers, samples were further processed. For 2 other samples, blood aggregates were visible upon arrival. These samples were sent to the same participating site during a period with atmospheric temperatures above 30 °C, the maximum recommended for correct stabilization in CellSave tubes, a factor that might have contributed to the poor condition of the samples. These were not further processed and were replaced by new samples sent in a new batch. Samples were processed at the analytical sites within 36 h after sample preparation. Spiked cells could be identified in 94 out of the remaining 96 samples processed (Fig. 3A, Table 2). The exceptions were 2 samples processed with the VyCAP Microsieves: one without stained cells and one with an abnormally high number of cells on the filter and nonspecific staining of CD45 on CK^pos cells. In some Siemens and VyCAP filters, the phenotypical evaluation of enriched cells, important for identification of intact cells, was compromised. This was likely due to the irregular surface of the filters (in the case of Siemens unit) and to the cellular distortions caused by the pressure necessary to make cells pass through the pores of the filters (in the case of both units). Standard automatic scanning of the irregular surface of the Siemens filters was not possible and was limited to one laboratory that was able to customize the scanning software for this application (Supplemental Table 1, participating site #5). In addition, automatic scanning of Parsortix^® was not practicable due to the large area and the depth of chamber in which cells are captured.

As expected, the EpCAM^low NCI-H1563 cells could not be detected by CellSearch^® and were therefore excluded from the ring experiment testing this system (Fig. 3B and C). Collectively for NCI-H441 cells, recoveries with the Parsortix^® (mean: 71%), CellSearch (mean: 75%), and RareCyte (mean: 68%) systems were significantly higher than with the Siemens (mean: 54%) and the VyCAP (mean: 49%) systems (P < 0.05 by Mann-Whitney U test).

For NCI-H1563 cells, recoveries with Parsortix^® (mean: 67%) and RareCyte (mean: 76%) were significantly higher than those with the VyCAP Microsieves (mean: 32%) (P < 0.01 by Mann-Whitney U test). Recovery with RareCyte was also significantly higher than that with Siemens (mean: 56%). We could not observe any statistically significant difference in the recovery of NCI-H441 and NCI-H1653 cells in the size-based technologies as expected due to the reduced differences in the cellular size between both lines. The smallest variance (measured as the standard deviation of all CTC enumerations done with a specific technology) was obtained for the Siemens filtration unit followed by the CellSearch^® system (Fig. 3D).

Discussion

In this multi-center study we established a platform enabling technical benchmarking of methods for CTC enrichment/isolation/detection. This platform was created within the frame of the European Innovative Medicines Initiative (IMI) consortium CANCER-ID (15) in which scientists from academic, clinical, and industrial sites in Europe and in the USA have joined forces to evaluate innovative technologies in the field of liquid biopsy. Currently, there is an increasing number of technologies available for CTC detection and their performance and reproducibility across multiple and real-life laboratories sites are very difficult to infer as all the data available is dispersed and quite difficult to compare. Facing this, this CANCER-ID platform was created to generate comparative data with multiple technologies and to be used as a tool for technology developers to assess the technical performance characteristics of their technologies and standard operating procedures (SOPs) in multiple academic sites using standardized samples. From the numerous CTC detection technologies commercially available, we selected methods from companies that were members of CANCER-ID or that approached us directly for testing their own technology.

This CANCER-ID proficiency testing platform organized ring trials to benchmark enrichment and detection of NSCLC cells spiked in healthy donor blood samples. It was designed to assess the between-laboratory reproducibility of the methods (rather their repeatability). With the 1-week interval between sample shipments, the values reported for each site represent the intermediate precision, and combining the intermediate precision results from all participating laboratories per platform, we determine the reproducibility (i.e., the precision between measurements obtained at different laboratories).

As preanalytical variables can account for more than 50% of the errors in general clinical laboratory testing (18), in these ring trials measures were taken to limit/eliminate the confounding impact that preanalytical variables could have on the results. To limit variability in cancer cell line culture conditions, blood sample draw, cell spiking, and sample transport, all samples were prepared in one same reference laboratory and shipped to different participating sites under similar transport conditions. To increase accuracy for cell-spiking, we chose direct enumeration over an uncontrolled dilution approach. However, irrespective the spiking-strategy, it is impossible to verify the exact cancer cell number in the blood sample, which may inevitably contribute to preanalytical variation. To mitigate any biological/biochemical effects by mixing allogenous cells and to guarantee optimal sample stabilization, cancer cell lines and blood samples were stabilized separately before spiking using the blood collection tubes indicated by the respective SOPs. Furthermore, in order to obtain uniform analytical conditions across the participating sites, staff received on-site training from the technology providers and could sufficiently test the feasibility of the SOPs.

For these experiments we have used 2 lung cancer cell lines with distinct phenotypic and molecular characteristics: NCI-H441 (EpCAM^high, CK^pos, KRAS^mut, TP53^mut, and EGFR^mut) and NCI-H1563 (EpCAM^low, CK^pos, KRAS^wt, TP53^wt, and EGFR^wt). As part of standard procedure, cells were confirmed to be mycoplasma-free just before the first shipment, and cell viability and size were controlled prior every shipment. Although these 2 cancer cell lines may not represent the whole tumor cell diversity observed in lung cancer patients, their different EpCAM expression levels constitute an opportunity to challenge the known fundamental principles of the chosen technologies. This is best reflected in the results obtained with the CellSearch^® system that could enrich EpCAM^high NCI-H441 cells but strikingly failed to enrich EpCAM^low NCI-H1563 cells, consistent with the dependency of this method on EpCAM expression previously demonstrated in various other cellular models and tumor entities (19–22). Among the panel of cell lines tested, we could not find a model with a cellular size small enough to challenge the limits of size-based technologies (Parsortix^®, VyCAP, and Siemens) with pore/gap sizes between 5 and 8 µm. As the cutoff of these technologies is clearly below the cell size of the 2 cell lines, the recovery for these 2 cell lines did not differ significantly in any of the platforms performing size-based enrichment. Additional smaller NSCLC cellular models could eventually help to validate the cutoffs of the technologies but these were not available.

The mean recovery rates for all tested systems ranged between 49% and 75% for NCI-H411 and 32% and 76% for NCI-H1563, respectively. Because of its inability to enrich EpCAM negative cells, CellSearch^® was excluded from the ring trial involving (EpCAM-negative) NCI-H1563 cells. CellSearch^®, Parsortix^®, and RareCyte all allowed higher recoveries, however, the 2 last systems displayed also the highest variance between measurements with NCI-H441 cells while enumeration with the CellSearch^® and the Siemens filtration unit were the most reproducible (Fig. 3D). Of note, in CellSearch^® and Siemens, enrichment and staining procedures are performed in a single streamlined automated protocol. Furthermore, the CellSearch^® system includes a dedicated detection unit. Differences in the light source, optical filters, and detection cameras, might account for the larger assay variances observed in some of the other technologies, despite the fact that the different participating groups have used microscope systems globally suitable to identify the fluorescent dyes used (Supplemental Table 1). In general, these results suggest the advantages of automated protocols and dedicated instrumentation covering the complete analytical workflow to increase technical reproducibility, a key aspect for introduction of a new technology in the clinics. Processing laboratories interested in applying any CTC enrichment/isolation technologies without dedicated detection system that were tested here (i.e., Parsortix^®, Siemens, and VyCAP) need extra efforts to verify the procedure on their in-house detection equipment, and eventually additional investment to adapt the enrichment/isolation procedure and/or the detection equipment.

For the laboratory routine, the practicability of sample management and the throughput of the assay are paramount. In this context, the BCT with fixation solution recommended for each assay is an important issue for flexible sample shipment and efficient sample processing. Although BCT fixatives increase sample stability, it should be noted that they compromise downstream gene expression analyses in CTCs (23). The storage times of the BCTs varied between 3 (AccuCyte^®) and 5 days (TransFix), which needs to be taken into consideration before routine use. Another important issue regarding laboratory practicability is the possibility to pause a workflow at a stable point. The majority of the workflows tested require all steps to be performed consecutively, including the microscope-based detection to avoid fading of the fluorescent dyes. An exception is RareCyte that allows long-term storage of processed samples at -20 °C (up to 1 year) prior to immunostaining, which can increase the flexibility for study design or sample management. The indicated maximum throughput of the assays (Table 1) was irrelevant for the proficiency testing but needs to be taken into consideration for clinical studies in real-life laboratories.

One other foreseeable important issue for implementation of any of these assays in the clinics is proper documentation of the results. In this aspect, the CellSearch^® and RareCyte systems are clearly ahead of all the other tested technologies by performing automatic microscopic scanning of the entire fraction of enriched cells and making possible subsequent software-assisted identification of CTCs by different users. In contrast, the uneven surface of the Whatmann filters used in the Siemens unit and the lack of dedicated detection systems able to deal with the specificities of the filters, forced the groups to perform a manual scanning which limits the documentation of the findings and their later verification.

Choosing the best method for CTC enrichment can be challenging and in this field, there is a lack of standardization and data from inter-laboratory testing raising questions on the technical reproducibility of the methods. Furthermore, comparison of results from different studies is hampered owing to different preanalytic conditions, protocols, cellular models, and patient samples, which also raises questions on the reproducibility of the results. The platform and the multi-national ring-trials established by CANCER-ID aimed to tackle some of these problems by using centrally prepared samples, consensus SOPs, and a unified reporting system. This ring-testing platform for CTCs does not intend to replace the efforts required from technology providers to prove the technical capabilities of their technologies, but rather to provide accountable information on assay reproducibility. Nevertheless, it can be adapted to test other relevant parameters of their technologies (e.g., intra-lab repeatability and linearity of recovery). Subsequently, it will be of utmost importance to clinically validate the technologies (if not done yet) in the context of clinical trials, which was beyond the scope of our project. This platform is not restricted to CANCER-ID partner companies as documented by the participation of RareCyte. In fact, this structure is meant to be useful to the broad community of technology providers who can thus compare the performance of their technologies in different real-life laboratories, as well as to the analytical sites for external quality assurance purposes. Furthermore, we believe that this proficiency testing program could help to define minimal requirements for performance qualification prior to clinical validation of technologies. For all these reasons, the program will remain in operation beyond the funding period of CANCER-ID. These activities will be sustained by the establishment of the European Liquid Biopsy Society (ELBS) (24) which comprises a large network of almost 100 institutions from academia and industry.

Supplemental Material

Supplemental material is available at Clinical Chemistry online.

Author Contributions

All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved.

R.P.L. Neves, statistical analysis; W. Ammerlaan, provision of study material or patients; E. Rossi, provision of study material or patients; F. Betsou, provision of study material or patients; F. Farace, administrative support; T. Schlange, administrative support; R. Zamarchi, statistical analysis; K. Pantel, financial support, administrative support; N.H. Stoecklein, financial support, administrative support.

Authors’ Disclosures or Potential Conflicts of Interest

Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest:

Employment or Leadership

M.V. Luetke-Eversloh, Bayer AG; T. Schlange, Bayer AG; S. Bender, Bayer AG.

Consultant or Advisory Role

K. Pantel, Menarini/Silicon Biosystems.

Stock Ownership

T. Schlange, Bayer AG; L.W.M.M. Terstappen, VyCAP.

Honoraria

None declared.

Research Funding

The authors participate in the Innovative Medicines Initiative consortium CANCER-ID. CANCER-ID is supported by the Innovative Medicines Initiative (IMI) Joint Undertaking under grant agreement No115749, resources of which are composed of financial contributions from the European Union’s Seventh Framework Program (FP7/2007–2013) and EFPIA companies’ in-kind contributions. The European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No765492 (H. Wikman, K. Pantel).

Expert Testimony

R.P.L. Neves, Thermo Fisher Scientific, Terumo BCT.

Patents

L.W.M.M. Terstappen, patents related to the CellSearch and VyCAP platform.

Other Remuneration

R.P.L. Neves, Menarini Silicon Biosystems, Thermo Fisher Scientific, Terumo BCT; N.H. Stoecklein, Menarini Silicon Biosystems.

Role of Sponsor

The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, preparation of manuscript, or final approval of manuscript.

Acknowledgments

The consortium thanks all blood donors, and Olga Kofanova and Kate Sokolowska (IBBL), Oliver Mauermann (UKE), Arjan Tibbe and Joska Broekmaat (VyCAP), Michael O’Brien and Lara Stevanato (ANGLE), Karen Marfurt, Michael Pugia and Guido Hennig (Siemens), Nicolò Manaresi and Tim Pitfield (Menarini Silicon Biosystems), and Tad George and Michael Rooney (Rarecyte) for their support.

References

Nonstandard Abbreviations

 
• CTCs

  circulating tumor cells

 
• EpCAM

  epithelial cell adhesion molecule

 
• EMT

  epithelial-to-mesenchymal transition

 
• FDA

  U.S. Food and Drug Administration

 
• SMEs

  small- and medium-sized enterprises

 
• IMI

  innovative medicines initiative

 
• SOPs

  standard operating procedures

 
• NSCLC

  non-small cell lung cancer

 
• ATCC

  American type culture collection

 
• CK

  cytokeratin

 
• PE

  phycoerythrin

 
• WGA

  whole genome amplification

 
• CNA

  copy number alterations

 
• BCT

  blood collection tube

 
• BSA

  bovine serum albumin

 
• PBS

  phosphate-buffered saline

 
• IBBL

  Integrated BioBank of Luxembourg

 
• ABC

  antibodies bound per cell

Author notes