A comparison of frailty measures in population-based data for patients with colorectal cancer

Abstract

Background

Numerous studies have revealed age-related inequalities in colorectal cancer care. Increasing levels of frailty in an ageing population may be contributing to this, but quantifying frailty in population-based studies is challenging.

Objective

To assess the feasibility, validity and reliability of the Hospital Frailty Risk Score (HFRS), the Secondary Care Administrative Records Frailty (SCARF) index and the frailty syndromes (FS) measures in a national colorectal cancer cohort.

Design

Retrospective population-based study using 136,008 patients with colorectal cancer treated within the English National Health Service.

Methods

Each measure was generated in the dataset to assess their feasibility. The diagnostic codes used in each measure were compared with those in the Charlson Comorbidity Index (CCI). Validity was assessed using the prevalence of frailty and relationship with 1-year survival. The Brier score and the c-statistic were used to assess performance and discriminative ability of models with included each measure.

Results

All measures demonstrated feasibility, validity and reliability. Diagnostic codes used in SCARF and CCI have considerable overlap. Prevalence of frailty determined by each differed; SCARF allocating 55.4% of the population to the lowest risk group compared with 85.1% (HFRS) and 81.2% (FS). HFRS and FS demonstrated the greatest difference in 1-year overall survival between those with the lowest and highest measured levels of frailty. Differences in model performance were marginal.

Conclusions

HFRS, SCARF and FS all have value in quantifying frailty in routine administrative health care datasets. The most suitable measure will depend on the context and requirements of each individual epidemiological study.

Key Points

• The three measures of frailty included classified differing proportions of the colorectal cancer population as frail.

• The prevalence of frailty increased with chronological age in all three measures.

• Higher levels of measured frailty were associated with higher rates of death within a year of colorectal cancer diagnosis.

• All three frailty measures included were able to quantify frailty and could be used more widely in epidemiological studies.

Introduction

Frailty is a concept which describes declines in multiple physiological systems, resulting in reduced resistance to stressors and increases risk of adverse clinical outcomes [1, 2]. As people age their risk of becoming frail increases [3], so does their risk of developing cancer [4]. Those who experience both face significant challenges, as frailty may limit the ability to receive potentially curative treatments, extends recovery times and increases the risk of morbidity and mortality [5–7].

Colorectal cancer (CRC) is a common cancer affecting over 42,000 people each year in the UK, over 40% of these will be over the age of 75 at diagnosis [8]. UK outcomes from CRC lag behind those attained by many comparable countries and the deficit is driven by poor outcomes amongst older people [9]. Drivers of these age-related inequities and inequalities are unclear [10, 11] and the contribution frailty makes is not well understood. Given its potential influence on management and outcome [7, 12], the ability to quantify frailty in population-level data, and incorporate it into epidemiological studies, is vital. To date, however, it has not been possible. As a result, many such studies are limited by simply considering age in their investigations of variation in care [13, 14] which, given older adults are a heterogeneous group in terms of health and fitness level, is inadequate.

Frailty is related to co-morbidity which has, to date, been more routinely considered. Co-morbidity can be defined as the simultaneous presence of two or more medical conditions and, like frailty, its incidence increases with age. Whilst frailty and co-morbidity are closely related they are different concepts and, ideally, should be considered separately [1, 15]. Particularly as studies have shown that the presence of increasing levels of co-morbidity alone do not account for the poor outcomes observed in older patients [16].

In the clinical setting there are a number of assessments that can be used to make an objective and robust quantification of frailty [2, 17–23]. Quantifying frailty in population-based administrative datasets that are used for epidemiological studies is more challenging. Several frailty measures have been developed for use in these datasets, but their relative strengths and weaknesses have not been explored in comparison with each other [24–26]. Given the scale of age-related inequalities and inequities in CRC care, there is an urgent need to better quantify frailty and assess its impact [3, 5, 7, 10, 11]. This study seeks to compare the utility of three commonly utilised indices of frailty which can be derived and applied in HES data in England: the Hospital Frailty Risk Score (HFRS) [24], Secondary Care Administrative Records Frailty (SCARF) index [25] and a frailty syndromes (FS) model [26].

Materials and methods

Data and study population

The utility of these measures was investigated in a national cohort of patients with CRC. This consisted of all individuals aged ≥18, diagnosed with the disease (International Classification of Diseases Version 10 (ICD10) codes [27] C18-C20) in England between 2016 and 2019. Linked cancer registration and Hospital Episode Statistics (HES) [28] data were obtained from the COloRECTal cancer data Repository (CORECT-R) [29]. Information used included age group (18–64, 65–74, 75–84 and ≥85 years), sex, tumour site (colon or rectum), tumour stage (I to IV or unknown) and survival time (in days calculated from date of diagnosis to date of death or censoring on 31 December 2020). Information was provided on all diagnostic codes included in the different frailty indices at attendances at National Health Service (NHS) hospitals derived from HES data in the 2 years prior to diagnosis.

Overview of frailty measures

Three frailty measures were investigated, all of which were created from, and designed for use in HES, an English hospital discharge dataset in which all diagnostic reasons for daycase and inpatient admissions are coded using ICD10 codes. Primary care data were not included in this project precluding those measures constructed from such data.

The first measure utilised was the HFRS [24]. This was developed from a HES dataset containing admissions of patients, with and without cancer, aged 75 years and over. ICD10 codes were given a weighted score proportional to the strength of prediction for hospital outcomes. The HFRS score is calculated by summing the score over all 109 codes for each individual. The categorisation of the HFRS for an individual was based on the maximum discrimination of the outcomes: low risk (HFRS <5), moderate risk (HFRS 5–15) and high risk (HFRS >15). In the development cohort utilised by Gilbert et al., 66.0% of individuals were classified as low risk, 20.3% as moderate and 13.7% as high risk [24].

The second was the SCARF index [25]. This was developed using a national cohort of women aged 50 years and over with oestrogen receptor positive early invasive breast cancer. This index includes codes for 32 health and function deficits. The SCARF index for each individual is calculated as the total number of deficits divided by the total in the index (n = 32). The following categorisation of the index is recommended: fit (0–1 deficit, index ≤0.05); mild frailty (2–3 deficits, index 0.06–0.11); moderate frailty (4–5 deficits, index 0.12–0.18); severe frailty (≥6 deficits, index ≥0.19).

The final measure was a risk prediction model for acute care [26], not restricted to individuals with cancer, based on a national cohort of individuals aged ≥65 years admitted as an emergency [26]. The authors classified patients across the following FS: anxiety and depression, cognitive impairment (including senility, dementia and delirium), functional dependence, falls and fractures, incontinence, mobility problems and pressure ulcers. To quantify the prevalence of frailty, the authors also used a count of the FS [30], which has been employed in this study (0 frailty syndromes, 1 frailty syndrome, 2 frailty syndromes and ≥3 frailty syndromes).

Statistical analysis

The three frailty measures were derived for each person in the cohort. The characteristics of the study population were assessed, both overall and in relation to their vital status (dead/alive) at 1 year from diagnosis. Due to the known relationship between age and frailty and the fact the measures were developed in different age profiles, analyses were repeated stratified by age (<75 & ≥75). This split was selected as 74 marks the end of the eligibility for routine CRC screening within the English NHS and this group has been shown to be the most rapidly growing proportion of the population in England [31].

To investigate the utility of the three frailty measures, their feasibility, validity and reliability were assessed following an approach previously used to compare co-morbidity measures [32]. The proportion of each population for which an indication of frailty could be determined was calculated in order to assess feasibility.

To assess the content and face validity, the prevalence of frailty as assessed by each measure across the whole cohort and by year of age at cancer diagnosis was assessed. As frailty has a strong inverse relationship with survival, 1-year Kaplan–Meier curves were estimated for each measure, both overall and stratified by age (<75 and ≥75). The log-rank test was used to test the equality of survivor functions across categorical levels of frailty for each measure.

Predictive validity was assessed by fitting a baseline logistic regression model with 1-year mortality as the outcome and age, sex, tumour site and tumour stage at diagnosis as predictors. These predictors were selected due to their known relationship with survival from CRC [33–38]. Each measure of frailty was then added to the baseline model to assess model performance. The scaled Brier score was used to assess accuracy of prediction, scaled by its maximum score to range between 0 and 100% (Brier_scaled = 1 – Brier/Brier_max, (where Brier_max = mean(p)*(1-p) and p = predicted probabilities), with higher values indicating better performance [39]. Discriminative ability was assessed using the concordance (c-) statistic. To improve estimates of the model performance statistics when performing internal validation, we used bootstrapping with 50 replications [40]. This was again replicated stratified by age (<75 and ≥75) to assess the validity in the younger and older populations.

The degree of overlap between the frailty measures and the routinely used Charlson Comorbidity Index (CCI) [41] was assessed. Given the interrelationship between co-morbidity and frailty, the diagnostic codes common to the CCI and each of the three frailty indices were identified. The number of common codes in the three frailty measures was, therefore, compared with those in the CCI.

Reliability was assessed by evaluating the consistency of the measures over time. The measures were calculated over a longer time period (2005–19) using the same inclusion criteria as stated before, allowing for assessment of any temporal changes in data. A descriptive analysis of the prevalence of frailty by year was undertaken by plotting the percentage of patients categorised annually as ‘fit’ (low-risk using HFRS, fit using SCARF or zero frailty syndromes using FS). To gauge expected changes in prevalence, we compared the trend in prevalence with that of the electronic Frailty Index (eFI) from published data [42].

All statistical analyses were undertaken in Stata 16.1.

Results

Study population

In total, 136,008 individuals diagnosed with CRC within the English NHS between 2016 and 2019 were included, with 36,255 (26.7%) dying within 1 year from the date of CRC diagnosis. The characteristics of this population are shown in Table 1.

Feasibility

It was feasible to create all the measures. In total, the measures used 435 different ICD10 codes to quantify frailty: HFRS used 109, SCARF 377 and FS 49 codes (Supplementary Table 1). A total of 134,192 (98.7%) could be linked to a HES record in the 2 years prior to their CRC diagnosis. Of those who had linked HES data, just over half, 54.2% (n = 73,778), had no indication of frailty from any of the three measures.

Validity

The prevalence of frailty varied across the three measures with 3.8% (n = 5210) of the study population classified as high-risk using the HFRS measure, 13.6% (n = 18,471) classified as having severe frailty using the SCARF measure and 2.1% (n = 2835) having ≥3 frailty syndromes using the FS model (Table 1). Across all three measures the proportion of individuals with a higher level of frailty at CRC diagnosis was greater amongst those who died within a year of their CRC diagnosis, compared with those who were alive at this point. Of those who died within a year of CRC diagnosis, 9.8% were classified as high risk using the HFRS, 25.7% as having severe frailty using SCARF and 5.6% as having ≥3 frailty syndromes using FS, compared with 1.7, 9.2 and 0.8% of those alive at 1-year (Table 1).

Across all three measures, the prevalence of frailty increased with age. In HFRS, 0.6% of those aged <65 were classified as high risk and this increased to 11.4% amongst those aged ≥85. Using SCARF, the proportion classified as having severe frailty increased from 3.3% in those aged <65 to 30.9% in those aged ≥85. For FS, the proportion with ≥3 syndromes increased from 0.4% in the <65 age group to 6.4% in those aged ≥85 (Figure 1 and Supplementary Table 2). Using the SCARF index, the prevalence of frailty was higher in the younger age groups compared with the other measures; the SCARF index classified 74.4% of patients aged 18–64 as fit, compared with 94.8% as low-risk in the HFRS and 89.5% with 0 frailty syndromes. In all three measures, the proportion of individuals classified as fit (low risk HFRS, fit SCARF or 0 frailty syndromes) decreased rapidly from the age of 70 (Figure 1).

The individuals classified as fit displayed comparable 1-year overall survival across the three measures (HFRS -78% (95%CI 78%–78%), SCARF—81% (95%CI 80%–81%), FS -78% (95%CI 78%–79%)) (Figure 2). HFRS and FS showed the greatest difference in 1-year overall survival between those with the lowest and highest levels of frailty, with a difference of >45% compared with 31% when using SCARF (Figure 2 and Table 2). Survival estimates were significantly different across categorical levels of frailty for all three measures (P < 0.01).

SCARF and HFRS had the largest overlap of codes, with 44 present in both, compared with 17 present in both SCARF and FS and none present in both HFRS and FS. Overall 20 codes were present in all three measures, with 302 codes only present in SCARF, 44 only in HFRS and 26 only in FS. All three frailty metrics had a degree of overlap with the CCI, the SCARF metric having the largest proportion of codes in common and FS having the least (Supplementary Table 3). A subset of the dementia component of the CCI was the only diagnoses present in all three frailty measures.

Predictive validity

Both the c-statistic and scaled Brier score increased after adding a frailty measure to the baseline model showing an improvement in the prediction of 1-year overall mortality (Table 3). This was seen in the population as a whole and when stratified by age (<75 and ≥ 75). Overall, the model including the HFRS measure performed better than models including either SCARF or FS, but the differences were marginal (Table 3).

Reliability

The percentage of patients categorised as fit decreased over time from 2005 to 2019 in the three measures (Supplementary Figure 1). Overall, the decrease was largest in SCARF (13.4 versus 8.8% in HFRS and 9.7% in FS). In all measures, larger decreases were observed in the older age groups and generally occurred in the early years of the study period. For comparison, published data on the eFI showed a decrease of 12.4% from 2006 to 2017, again with the largest decreases observed in the older age groups.

Discussion

This study compared three different methods designed to quantify frailty using population-level administrative hospital data across a population of people diagnosed with CRC and has shown all have value. All were feasible to create, appeared to be valid, predicted short-term overall survival and were reliable. Each had different strengths and weaknesses which should be considered when deploying them in population-based epidemiological studies.

The prevalence of frailty differed substantially across the three measures. SCARF had the widest discrimination across the population with 55.4% of patients being deemed fit and 45.6% having some form of frailty. In contrast, HFRS and FS classified 85.1 and 81.2% in their lowest risk/frailty categories, respectively. This range in prevalence reflects the fact that frailty syndrome is a continuum whose severity is assessed on the basis of a constellation of conditions. The cut-off for the diagnosis of frailty using a particular score depends on the conditions assessed and the weight they are given.

Each of the measures was categorised differently with HFRS classifying patients into low, intermediate and high risk, SCARF into fit, mild, moderate or severe frailty and FS by number of frailty syndromes. This makes direct comparison across measures challenging, but may be relevant when considering which index to use in a particular analysis dependent of the research question. For example, FS may be preferable when specific frailty deficits are of interest in study.

The tools studied in this work were not initially developed for CRC patients. However, because conditions included are not specific to a single population, we believe in their values as frailty indicator to be used in population-based studies. This study was not intended to develop or adapt a frailty score for specific use in the CRC population, rather to test the feasibility of using existing scores in this population. The measures compared in this study are not designed for use in a clinical setting, rather to account for frailty in research using population-level data, contributing to hospital benchmarking and to inform public health interventions. The ability to quantify frailty outside of a clinical setting has implications for the management of CRC. Recent population-based studies have reported differing surgical treatment strategies in relation to age across multiple countries [13], and variation in the use of adjuvant chemotherapy in relation to age within the English NHS [14]. There is also evidence that cancer multidisciplinary team decisions are not implemented in a significant proportion of patients and patients being unfit for treatment may be responsible [43]. The addition of a frailty measure would allow for an assessment of fitness within these populations. Understanding the impact of frailty at a population level is part of working out why some patients do not receive what would seem to be the optimal treatment for their cancer.

A particular challenge when investigating frailty is distinguishing it from the related, but distinct, concept of co-morbidity. Comparisons of the codes used in the commonly used CCI with the frailty measures investigated in this study revealed considerable overlap for SCARF that would preclude using it alongside the CCI in an analysis, depending on the outcome in question SCARF or CCI could be utilised independently but the use of both together should be avoided. This reflects the challenges of frailty, and co-morbidity, assessment in epidemiological studies where the majority of perioperative clinical risk scores are based on the presence of absence or diagnostic codes [44, 45].

This challenge of quantifying frailty through diagnostic codes alone is, perhaps, the biggest limitation of all the frailty measures investigated. This study utilised diagnostic codes recorded during admissions in the 2-years preceding CRC diagnosis, therefore individuals who did not have an admission in that time period would be reported as not frail. Additionally, only the presence or absence of a condition was recorded so there was no indication of severity of condition. Frailty is a complex syndrome and reducing its quantification to the presence of absence of particular diagnoses in hospital attendances alone may be too simplistic. Variation in recording and coding of inpatient admissions over time may also introduce error into the measurement of individual components of frailty scores. For this reason, we limited the main analyses in this study to a 4-year period. In the extended analyses covering a longer time-period the measures displayed similar patterns of frailty prevalence to that of the eFI.

A further limitation of this study was that only measures constructed from secondary care data were considered. Stronger quantification of frailty would be expected from indices such as the eFI [46], or the preoperative frailty index [47] which include additional data sources such as primary care and prescription databases. Inevitably, measures captured through direct clinical assessment would provide the gold standard. This information is not routinely available to population-based epidemiological studies and, in their absence, HFRS, SCARF and FS provide a means to quantify frailty. Although that quantification can only ever be as complex and in depth as the data they are derived from, they still offer an important mechanism for considering its association with outcomes in descriptive epidemiological studies. The absence of a gold-standard measure of frailty to compare the measures used in this study can also be considered a limitation. However we compared each measure using an adverse outcome in the same way the measures were developed. The relationship with mortality is an important and non-arbitrary test of predictive validity [48].

HFRS, SCARF and FS already offer a mechanism for quantifying frailty and enabling its consideration in epidemiological studies. Given the growing evidence of persistent, and significant, age-related CRC inequities, and the uncertain contribution frailty may make in driving them, they are a valuable tool. Future studies should deploy them effectively to generate evidence that will help both identify, and eliminate, age-related inequities in CRC care and outcomes across the entire patient journey, and could seek to compare the association between frailty and treatment use and outcomes in different treatment pathways.

Acknowledgements

This work uses data that have been provided by patients and collected by the NHS as part of their care and support. The data are collated, maintained and quality assured by the National Cancer Registration and Analysis Service. Access to the data was facilitated by the NHS England’s Data Access Request Service and the Cancer Research UK funded UK Colorectal Cancer Intelligence Hub (C23434/A23706).

Declaration of Conflicts of Interest

None.

Declaration of Sources of Funding

Cancer Research UK (grant C23434/A23706), Yorkshire Cancer Research (grant L394). This work was supported in part by the Leeds National Institute for Health and Care Research (NIHR203331) Biomedical Research Centre and the Oxford NIHR Biomedical Research Centre (NIHR203311). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. S.P. is supported by the Luxembourg National Research Fund (FNR), Project n° 16731054.

Ethical approval

23/SW/0069—South West—Central Bristol Research Ethics Committee.

References

Author notes