Automated vessel-specific coronary artery calcification quantification with deep learning in a large multi-centre registry

Abstract

Aims

Vessel-specific coronary artery calcification (CAC) is additive to global CAC for prognostic assessment. We assessed accuracy and prognostic implications of vessel-specific automated deep learning (DL) CAC analysis on electrocardiogram (ECG) gated and attenuation correction (AC) computed tomography (CT) in a large multi-centre registry.

Methods and results

Vessel-specific CAC was assessed in the left main/left anterior descending (LM/LAD), left circumflex (LCX), and right coronary artery (RCA) using a DL model trained on 3000 gated CT and tested on 2094 gated CT and 5969 non-gated AC CT. Vessel-specific agreement was assessed with linear weighted Cohen’s Kappa for CAC zero, 1–100, 101–400, and >400 Agatston units (AU). Risk of major adverse cardiovascular events (MACE) was assessed during 2.4 ± 1.4 years follow-up, with hazard ratios (HR) and 95% confidence intervals (CI). There was strong to excellent agreement between DL and expert ground truth for CAC in LM/LAD, LCX and RCA on gated CT [0.90 (95% CI 0.89 to 0.92); 0.70 (0.68 to 0.73); 0.79 (0.77 to 0.81)] and AC CT [0.78 (0.77 to 0.80); 0.60 (0.58 to 0.62); 0.70 (0.68 to 0.71)]. MACE occurred in 242 (12%) undergoing gated CT and 841(14%) of undergoing AC CT. LM/LAD CAC >400 AU was associated with the highest risk of MACE on gated (HR 12.0, 95% CI 7.96, 18.0, P < 0.001) and AC CT (HR 4.21, 95% CI 3.48, 5.08, P < 0.001).

Conclusion

Vessel-specific CAC assessment with DL can be performed accurately and rapidly on gated CT and AC CT and provides important prognostic information.

Graphical Abstract

Vessel-specific coronary artery calcium (CAC) from electrocardiogram gated and attenuation correction CT patients in a large multi-centre registry with Deep Learning can be performed accurately, rapidly, and provides important prognostic information for Major Adverse Cardiac Events (MACE).

Open in new tabDownload slide

Introduction

Cardiovascular disease (CVD) is the leading cause of morbidity and mortality worldwide¹ and there is an important need for methods that facilitate early detection and risk stratification. Coronary artery calcification (CAC) is a marker of the presence of atherosclerosis. CAC can be identified on computed tomography (CT), including electrocardiogram (ECG) gated cardiac CT, non-gated thoracic CT, low dose lung cancer screening CT, and non-gated attenuation correction (AC) CT performed during single photon emission tomography (SPECT) or positron emission tomography (PET) imaging. The presence of CAC is associated with an increased risk of future cardiovascular events in both symptomatic and asymptomatic patients.^2,3 In addition, the vascular distribution of CAC has been shown to have important prognostic implications over and above the global CAC score.^4,5

Recently deep learning (DL), a subtype of artificial intelligence, has shown great promise for the identification of CAC on CT. Previous studies have shown the ability of a variety of DL models to identify and quantify CAC on several types of CT.⁶ We have previously demonstrated that a novel convolutional long short-term model (convLSTM), which integrates information from adjacent image slices, can perform excellent per-patient CAC assessment on AC CT and has important prognostic implications.^7,8 Most previous studies have focused on the total per-patient Agatston CAC score. However, an important aspect of CAC assessment is where in the coronary artery tree the CAC is present. Vessel-specific CAC quantification with DL has previously been developed for ECG-gated CT and non-gated thoracic CT.^9,10 However, CAC on AC CT can provide important prognostic information and there is a need for machine learning models that are capable of vessel-specific CAC quantification on a wider range of types of CT scans, including AC CT.

The aim of this study was therefore to develop a DL model capable of vessel-specific CAC quantification on both ECG gated CT and non-gated AC CT, and to explore its prognostic implications.

Methods

Study design and patient population

In this multi-centre study, we used imaging and clinical data from four centres to train and test a DL convLSTM model capable of identifying and quantifying vessel-specific CAC scores. This included internal training (n = 2500) and validation (n = 500) on ECG gated non-contrast cardiac CT from Cedars Sinai Medical Centre (Los Angeles, US). External testing was performed on consecutive ECG gated non-contrast cardiac CT (n = 2094) from Cardiovascular Imaging Technologies (Kansas City, US) and on consecutive non-contrast AC CT (n = 5969) from patients undergoing SPECT/CT myocardial perfusion imaging in Yale (New Haven, US) and University of Calgary (Calgary, Canada). Patients with previous myocardial infarction, percutaneous coronary intervention, or coronary artery bypass grafts were excluded. This study was approved by the institutional review board at each site and was performed in accordance with the Declaration of Helsinki. To the extent allowed by data sharing agreements and institutional review board protocols, data, and codes used in this manuscript will be shared upon written request.

Image acquisition and image analysis

Detailed information on image acquisition has been previously published.^11–13 Briefly, ECG gated CT was performed during suspended end-inspiration with prospective gating (tube voltage 120 kVp, tube current-time product 85–150 mAs, 3 mm slice thickness). AC CT was performed during normal breathing, without ECG gating (tube voltage 100 kVp, tube current-time product 11–13 mAs, 3 mm slice thickness or tube voltage 120 kVp, tube current 20 mA, 5 mm slice thickness or 120 kV, 2.5 mm slice thickness, 60–150 mA).

Quantification of CAC was performed using dedicated software (QPET Suite, Cedars Sinai Medical Centre, Los Angeles) by two experts with at least 5 years of experience. Agatston CAC scores were calculated on a per-vessel basis using 130 Hounsfield unit threshold for analysing both ECG-gated and AC CT.¹⁴ The ground truth for model training and model testing was the expert derived Agatston CAC scores. CAC in the left main stem (LM), left anterior descending (LAD) coronary artery and diagonal arteries were assigned to the LM/LAD territory. CAC in the left circumflex (LCX) coronary artery and obtuse marginal branches was assigned to the LCX territory. CAC in the right coronary artery (RCA) and acute marginal branches was assigned to the RCA territory. CAC in the posterior descending artery and posterior lateral branches was assigned to the RCA territory unless it could be established from the images that the circulation was left dominant.

Deep learning model architecture and training

The model was constructed using PyTorch (Version 3.7.4) and comprised of two DL networks. The first DL network segmented vessel-specific CAC and the second created an ovoid segmentation to encompass the cardiac outline (Figure 1). CAC from the first network was assessed within the cardiac segmentation from the second network. We employed convLSTM models with three slices provided to each network as input.⁷ This method replicates the multi-slice assessment usually performed during manual image analysis and has been shown to reduce memory requirements and inference times compared to other commonly used DL networks.¹⁵

External testing

The DL model was tested on two types of external data including ECG gated non-contrast cardiac CT (External testing 1) and non-contrast AC CT (External testing 2) which did not form part of the training dataset. Per-vessel CAC scores for the manual ground truth and DL model were categorized into four groups (0, 1–100, 101–400, and >400 Agatston units). To determine reasons for discordant ground truth and DL scores 2.4% (n = 195/8054) of images were randomly selected for manual review.

Clinical information and outcomes

Information on cardiovascular risk factors and clinical outcomes was obtained from review of electronic health care records. Major adverse cardiovascular events (MACE) included death, revascularization, myocardial infarction, or admission for unstable angina. An additional definition of MACE was also assessed where early revascularization within 90 days of myocardial perfusion imaging was excluded. Experienced clinicians at each site adjudicated outcomes using standard criteria.¹³

Statistical analysis

Statistics analysis was performed using R (Version 4.2.3, R Foundation for Statistical Computing, Vienna, Austria) and Python 3.7.4. Continuous variables are presented as mean and standard deviation for normally distributed data and median and interquartile range (IQR) for non-normally distributed data. Categorical variables are presented as frequency and percentage. Agreement for ordinal was assessed using linear weighted Cohen’s Kappa and concordance correlation coefficients. Correlation coefficients and Cohen’s kappa were interpreted as <0.2 as very weak, 0.2 to <0.40 as weak, 0.40 to <0.60 as moderate, 0.6 to <0.80 as strong, and 0.8–1 as excellent. Statistical significance was assessed with Wilcoxon rank-sum test and Pearson’s Chi-squared test. The per vessel diagnostic accuracy for the identification of CAC of the DL model compared to the expert ground truth was assessed using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). A two tailed P-value <0.05 was considered statistically significant.

Results

Study population

The DL model was externally tested on 2094 ECG gated non-contrast cardiac CT and 5969 AC CT. Characteristics of training and external testing sets are shown in Table 1. The median ground truth expert CAC score was 1 [IQR 0, 64] AU in the training dataset, 30 [IQR 0, 327] AU in the external testing ECG gated CT dataset (External testing 1), and 19 [IQR 0, 261] AU in the external testing AC CT dataset (External testing 2).

Vessel-specific DL CAC assessment

Assessment of vessel-specific DL CAC scoring (Figures 2 and 3) took less than 6 s per scan using a graphics processing unit (GeForce RTX 2080, AMD Ryzen 9 59505950X 16-core Processors, 64 GB RAM).

For external testing on ECG gated non-contrast cardiac CT (n = 2094, External testing 1) there was excellent concordance (0.80 to 0.89) between DL and expert ground truth vessel-specific CAC scores (see Supplementary data online, Figure S1). Linear weighted Cohen’s Kappa showed excellent agreement for LM/LAD CAC (0.90, 95% CI 0.89, 0.92), and RCA CAC (0.79, 95% CI 0.77, 0.81), and strong agreement for LCX CAC (0.70, 95% CI 0.68, 0.73). For the identification of any CAC on ECG gated CT the sensitivity, specificity, PPV, and NPV for the DL model compared to expert ground truth was 92, 96, 97, and 88% for the LM/LAD, 84, 85, 76, and 90% for the LCX, and 89, 79, 78, and 90% for the RCA.

For external testing on AC CT (n = 5969, External testing 2) there was strong to excellent concordance (0.75–0.80) between DL and expert ground truth assessment of vessel-specific coronary calcium scores (see Supplementary data online, Figure S2). Linear weighted Cohen’s Kappa showed strong agreement for LM/LAD CAC (0.78, 95% CI 0.77, 0.80), LCX CAC (0.60, 95% CI 0.58, 0.62) and RCA CAC (0.70, 95% CI 0.68, 0.71). For the identification of any CAC on AC CT the sensitivity, specificity, PPV, and NPV for DL model compared to expert ground truth was 83, 93, 94, and 82% for the LM/LAD, 73, 84, 71, and 86% for the LCX, and 84, 80, 70, and 90% for the RCA.

Reasons for discordant DL and ground truth assessment of CAC included misclassification of mitral valve calcification as LCX calcification, high attenuation material such as prosthetic valves or pacemakers, motion artefact, pericardial calcification, and challenges in defining coronary osteal vs. aortic calcification (see Supplementary data online, Figure S3).

Impact of sex on DL CAC assessment

In the external testing datasets, there were 1151 (55%) female patients who had ECG gated CT and 2974 (50%) female patients who had attenuation CT. Agreement and concordance of vessel-specific DL CAC compared to expert ground (Figure 4) truth was similar, but slightly lower, in female compared to male patients for ECG gated CT (see Supplementary data online, Figure S4) and AC CT (see Supplementary data online, Figure S5). This effect was more prominent in AC CT and in the LCX.

Prognostic implications of vessel-specific DL CAC

Mean follow-up time of the external testing datasets was 2.4 ± 1.4 years [2.7 ± 1.6 years for ECG gated CT (External testing 1), 2.3 ± 1.4 for AC CT (External testing 2)] and MACE occurred in 242 (12%) of patients undergoing ECG gated CT and 841 (14%) of patients undergoing AC CT. On ECG gated CT (Figure 5, External testing 1) the highest risk of MACE was associated with DL LM/LAD CAC >400 AU (HR 12.0, 95% CI 7.96–18.0, P < 0.001), followed by DL LCX CAC >400 AU (HR 10.3, 95% CI 6.69–16, P < 0.001), and DL RCA CAC >400 AU (HR 8.23, 95% CI 5.69–11.9, P < 0.001), compared to patients with no CAC. On AC CT (Figure 6, External testing 2) the highest risk of MACE was associated with DL LM/LAD CAC >400 AU (HR 4.21, 95% CI 3.48–5.08, P < 0.001), followed by DL RCA CAC >400 AU (HR 4.05, 95% CI 3.34–4.92, P < 0.001, and DL LCX CAC >400 AU (HR 3.63, 95% CI 2.76–4.79, P < 0.001), compared to patients with no CAC. Vessel-specific CAC assessed by DL was a predictor of MACE, independent of age, sex, body mass index, hypertension, diabetes mellitus, dyslipidemia, family history, and smoking status in multi-variable analysis (Table 2, Figure 7). Early revascularization occurred in 87 (4.2%) of patients who underwent ECG gated CT (External testing 1) and 265 (4.4%) of patients who underwent AC CT (External testing 2). When patients with early revascularization were excluded from the MACE definition, the vessel-specific CAC assessed by DL remained an independent predictor of outcomes (see Supplementary data online, Table S1).

Discussion

In this study, we have developed a DL model which is capable of vessel-specific CAC assessment and demonstrated that it can rapidly and accurately assess CAC in large multi-centre external datasets including both ECG-gated and AC CT. The DL model showed strong to excellent agreement and concordance for the assessment of CAC on ECG gated non-contrast cardiac CT and low dose AC CT performed during SPECT myocardial perfusion imaging. Importantly we have demonstrated similarities and differences in performance in male and female patients and the downstream prognostic implications. Vessel-specific DL CAC was an independent predictor of subsequent MACE, with LM/LAD CAC >400 AU associated with the worst outcomes. This DL model will speed up the assessment of CAC in clinical practice and significantly reduce reporting time for clinicians.

Automated assessment of CAC is valuable for the integration of this assessment into routine clinical practice, in particular for AC CT. Manually assessed CAC has been shown to have important prognostic implications in multiple cohorts who have undergone different types of CT, including ECG-gated cardiac CT, routine thoracic CT, low dose lung cancer CT, and AC CT performed for during SPECT or PET imaging.^16–20 For patients undergoing SPECT myocardial perfusion imaging, information on CAC is additive to the prognostic information provided by myocardial perfusion imaging.^20–22 However, manual quantification is time consuming and so CAC is often not quantified or reported.²³ In this study we have developed and tested a DL model that is capable of rapidly and accurately assessing vessel-specific CAC on a variety of types of CT scans from multiple centres. This will provide valuable information for clinicians and make reporting faster.

Automated detection of CAC on AC CT for SPECT or PET imaging will enable the rapid integration of this additional information into the interpretation of myocardial perfusion imaging studies. AC CT is typically acquired with low radiation dose, meaning poorer image quality with more image noise compared to standard CT. It is therefore important that our DL model performed well on both AC CT and standard ECG gated CT. The diverse testing data used in this study is important to ensure the broad clinical applicability of this DL model. The model performed better in ECG-gated CT than attenuation CT, likely because of differences in image quality due to the different acquisition parameters and differences in clinical characteristics between the cohorts. It is important that we have tested this model in this range of settings and have demonstrated good performance, as this increases its potential generalizability to other populations and clinical scenarios. Automation of the time-consuming task of calcium scoring will improve the incorporation of this information into clinical practice and save valuable reporting time for clinicians. Other AI methods to assess CAC in small studies of AC CT have shown varying agreement for the assessment of total CAC (linear weighted Cohen kappa 0.70–0.82).^24–27 We have previously shown that a convLSTM DL model can be used to accurately assess total CAC on AC CT and that this information has prognostic value in patients undergoing PET and SPECT myocardial perfusion imaging.^8,12 The current study extends these results and highlights the importance of vessel-specific CAC for prognostic assessment.

Previous studies of DL models for CAC identification and quantification have primarily focused on the total CAC score. This ignores the important information provided by identifying the regional distribution of CAC which has been shown with manual analysis in large registry studies.^28,29 In our current study CAC within the LM/LAD provided the most important prognostic information, but valuable information was provided by all vessels. Some small previous studies have shown the potential for automated vessel-specific CAC assessment using DL.^30–32 Winkel et al tested a convolutional neural network on 262 ECG gated CT and showed a 94% accuracy for the identification of the correct branch, but vessel-specific calcium scores were not assessed.⁹ In non-gated lung cancer screening CT de Vos et al. used location specific CAC along with self-reported cardiovascular risk factors to predict CVD mortality with an area under the operator curve (AUC) of 0.76.¹⁰ Our paired machine learning models ensure that ascending thoracic aorta, arch of the aorta and descending thoracic aorta calcifications are not accidentally included in the assessment of CAC, as these are excluded from the cardiac silhouette created by the first machine learning model.

This study has some limitations which should be acknowledged. The training and testing datasets were large and heterogenous, but further testing on different types of scanners and reconstruction may provide further information. The DL model was better at stratifying higher CAC scores than the low/no CAC groups, particularly in the noisier low dose AC CT. In addition, in our dataset’s patients with no or low CAC were more prevalent than those with higher CAC groups. This can also be challenging for expert observers, so misclassification in the expert ground truth CAC assessment is also possible. Further developments of the machine learning model will include the assessment of non-coronary calcifications such as valvular calcification. As this was a retrospective analysis, the occurrence of MACE could have been influenced by medical and other management decisions which may have been based on the imaging findings. We excluded patients with previous coronary interventions, so this DL model should be used with care in such patients.

In conclusion, we have shown that a DL model can accurately and rapidly assess vessel-specific CAC on a variety of types of CT including ECG gated cardiac CT and AC CT performed during SPECT myocardial perfusion imaging. This automatic vessel-specific CAC assessment provided valuable prognostic information on both ECG-gated and AC CT scans which could be used to guide patient management.

Supplementary data

Supplementary data are available at European Heart Journal - Cardiovascular Imaging online.

Funding

This research was supported in part by grants R01HL089765 and R35HL161195 from the National Heart, Lung, and Blood Institute/National Institutes of Health (NHLBI/NIH) (PI: Piotr Slomka). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. M.C.W. (FS/ICRF/20/26002) is supported by the British Heart Foundation. A.M.M. is supported by a research scholarship from the Polish National Agency for Academic Exchange.

Consent

Written informed consent was waived by the institutional review board due to the retrospective nature of the study.

Data availability

To the extent allowed by data sharing agreements and institutional review board protocols, data and codes used in this manuscript will be shared upon written request.

References

Author notes