Abstract
In a large cluster-randomized trial on the impact of a prediction model, presenting the calculated risk of postoperative nausea and vomiting (PONV) on-screen (assistive approach) increased the administration of risk-dependent PONV prophylaxis by anaesthetists. This change in therapeutic decision-making did not improve the patient outcome; that is, the incidence of PONV. The present study aimed to quantify the effects of adding a specific therapeutic recommendation to the predicted risk (directive approach) on PONV prophylaxis decision-making and the incidence of PONV.
A prospective before–after study was conducted in 1483 elective surgical inpatients. The before-period included care-as-usual and the after-period included the directive risk-based (intervention) strategy. Risk-dependent effects on the administered number of prophylactic antiemetics and incidence of PONV were analysed by mixed-effects regression analysis.
During the intervention period anaesthetists administered 0.5 [95% confidence intervals (CIs): 0.4–0.6] more antiemetics for patients identified as being at greater risk of PONV. This directive approach led to a reduction in PONV [odds ratio (OR): 0.60, 95% CI: 0.43–0.83], with an even greater reduction in PONV in high-risk patients (OR: 0.45, 95% CI: 0.28–0.72).
Anaesthetists administered more prophylactic antiemetics when a directive approach was used for risk-tailored intervention compared with care-as-usual. In contrast to the previously studied assistive approach, the increase in PONV prophylaxis now resulted in a lower PONV incidence, particularly in high-risk patients. When one aims for a truly ‘PONV-free hospital’, a more liberal use of prophylactic antiemetics must be accepted and lower-risk thresholds should be set for the actionable recommendations.
Risk model-guided changes in clinician decision-making should not only document a change in practice but also better patient outcomes.
PONV prediction models have, at best, modest predictive utility for individual patients.
It is unclear whether a risk-modified strategy of PONV prophylaxis can meaningfully reduce rates of PONV.
The overall cost and net effect of universal PONV prophylaxis on patient comfort after surgery deserves further study.
Current guidelines on prevention of postoperative nausea and vomiting (PONV) recommend using risk-dependent strategies, where administration of antiemetic prophylaxis is based on individual risk using a prediction model.1,2 Although several prediction models have been developed,3–6 their effect on clinical practice is small, mainly because of poor implementation.7,8 These disappointing results of risk-dependent PONV prophylaxis have fostered ongoing debate as to whether or not to shift to routine PONV prophylaxis in all patients, irrespective of their predicted risk.9–11 Before switching to such a new, as yet unproven, strategy of administering multiple antiemetics to every patient, the impact of risk-dependent strategies for PONV should be critically evaluated.12–15 However, comparative studies assessing the actual impact of risk-dependent prophylaxis on the incidence of PONV are rare.16
We have previously shown that assisting anaesthetists by only presenting the patient's calculated risk of PONV on-screen in an anaesthesia information management system, but without further therapeutic directives per predicted risk, increased the number of prophylactic antiemetics administered by anaesthetists.17 However, this change in physician decision-making did not decrease the incidence of PONV. We hypothesized that a greater impact could be achieved when being more directive by simply adding actionable recommendations according to the calculated risks.12–14,18–20
The present before–after study aimed to quantify the effects of combining a specific therapeutic recommendation with the patient's predicted risk on both the incidence of PONV and the actual administration of risk-dependent PONV prophylaxis.
Methods
Design and participants
The present study was a prospective before–after cohort study, conducted at the Anaesthesiology Department of a Dutch university hospital (UMC Utrecht) in 2010. The study aimed to quantify the effects of a directive PONV prediction model approach (i.e. presenting predicted risks accompanied with non-obligatory, therapeutic recommendations) on both the incidence of PONV and the administration of antiemetic prophylaxis. Care-as-usual (see below) was studied during the before-period (January to March 2010), followed by an intervention period (April to May 2010), during which all physicians were provided with a recommendation on how many prophylactic antiemetics would be required to sufficiently lower their individual patients' PONV risks (see below).
According to Dutch law, research protocols that do not subject patients to a particular treatment or that require them to behave in a particular way, do not apply to the Medical Research Involving Human Subjects Act. As the decision support tools in our study protocol only provided evidence-based information to physicians, the institutional ethical review board waived the need for individual informed consent and approved the study protocol (Medical Ethics Review Board, UMC Utrecht, 11–553).
All adult patients undergoing general anaesthesia for elective, non-ambulatory surgery who had visited the outpatient preanaesthesia evaluation clinic were considered eligible for the study. Exclusion criteria were pregnancy, postoperative admission to the intensive care unit, overnight ventilation at the post-anaesthesia care unit, and inability to communicate in Dutch or English. All eligible patients from the time of study initiation were automatically included using the anaesthesia information management system.
The prediction model
The implemented prediction model was originally developed in a population of a different university hospital in the Netherlands and had already been externally validated.21,22 The model was subsequently updated for implementation at the UMC, Utrecht.23 The model consisted of seven predictor variables: age; gender; current smoking; type of surgery; inhalation anaesthesia (including nitrous oxide); ambulatory surgery; and history of motion sickness or PONV (Table 1).
The implemented prediction model for PONV. Probability of PONV as estimated by the model=1/{1+exp [−(0.12–0.017×age+0.36×female gender –0.50×current smoking+0.60×history of PONV or motion sickness+0.48×surgery with a high PONV risk+0.35×inhalation anaesthesia–1.16×outpatient surgery)]}. PONV, postoperative nausea and vomiting. *Definition of this predictor was ‘abdominal or middle ear surgery′. †When compared with i.v. anaesthesia using propofol. ‡Although the prediction model included a predictor for outpatient surgery, only non-ambulatory patients were included in the present study. Therefore, for each patient included in the study, this predictor was automatically set to zero. §For the intercept, the column represents the baseline odds not the OR
| Predictor | Updated model |
|---|---|
| OR (95% CI) | |
| Age (yr) | 0.98 (0.98–0.99) |
| Female gender | 1.44 (1.14–1.82) |
| Current smoking | 0.61 (0.48–0.77) |
| History of PONV/motion sickness | 1.82 (1.44–2.31) |
| Surgery with a high PONV risk* | 1.62 (1.14–2.30) |
| Inhalation anaesthesia† | 1.42 (1.12–1.79) |
| Outpatient surgery‡ | 0.31 (0.24–0.41) |
| Intercept§ | 1.13 (0.73–1.74) |
| Predictor | Updated model |
|---|---|
| OR (95% CI) | |
| Age (yr) | 0.98 (0.98–0.99) |
| Female gender | 1.44 (1.14–1.82) |
| Current smoking | 0.61 (0.48–0.77) |
| History of PONV/motion sickness | 1.82 (1.44–2.31) |
| Surgery with a high PONV risk* | 1.62 (1.14–2.30) |
| Inhalation anaesthesia† | 1.42 (1.12–1.79) |
| Outpatient surgery‡ | 0.31 (0.24–0.41) |
| Intercept§ | 1.13 (0.73–1.74) |
The implemented prediction model for PONV. Probability of PONV as estimated by the model=1/{1+exp [−(0.12–0.017×age+0.36×female gender –0.50×current smoking+0.60×history of PONV or motion sickness+0.48×surgery with a high PONV risk+0.35×inhalation anaesthesia–1.16×outpatient surgery)]}. PONV, postoperative nausea and vomiting. *Definition of this predictor was ‘abdominal or middle ear surgery′. †When compared with i.v. anaesthesia using propofol. ‡Although the prediction model included a predictor for outpatient surgery, only non-ambulatory patients were included in the present study. Therefore, for each patient included in the study, this predictor was automatically set to zero. §For the intercept, the column represents the baseline odds not the OR
| Predictor | Updated model |
|---|---|
| OR (95% CI) | |
| Age (yr) | 0.98 (0.98–0.99) |
| Female gender | 1.44 (1.14–1.82) |
| Current smoking | 0.61 (0.48–0.77) |
| History of PONV/motion sickness | 1.82 (1.44–2.31) |
| Surgery with a high PONV risk* | 1.62 (1.14–2.30) |
| Inhalation anaesthesia† | 1.42 (1.12–1.79) |
| Outpatient surgery‡ | 0.31 (0.24–0.41) |
| Intercept§ | 1.13 (0.73–1.74) |
| Predictor | Updated model |
|---|---|
| OR (95% CI) | |
| Age (yr) | 0.98 (0.98–0.99) |
| Female gender | 1.44 (1.14–1.82) |
| Current smoking | 0.61 (0.48–0.77) |
| History of PONV/motion sickness | 1.82 (1.44–2.31) |
| Surgery with a high PONV risk* | 1.62 (1.14–2.30) |
| Inhalation anaesthesia† | 1.42 (1.12–1.79) |
| Outpatient surgery‡ | 0.31 (0.24–0.41) |
| Intercept§ | 1.13 (0.73–1.74) |
Intervention
Care-as-usual group
During the care-as-usual period, anaesthetists were not exposed to any automated prognostic information by the prediction model. The prophylactic management of PONV was not standardized in any way, which was according to care-as-usual in our hospital. At that time, the existing, local protocol for administration of PONV prophylaxis only included a preferable order for antiemetic drugs, their dosage and timing of administration: (i) ondansetron 4 mg i.v., 30 min before emergence of anaesthesia; (ii) droperidol 1.25 mg i.v., 30 min before emergence of anaesthesia; (iii) dexamethasone 4 mg i.v., after induction of anaesthesia. Other prophylactic antiemetic drugs, such as NK-1 receptor antagonists, were not readily available in our hospital during the study period.
In the post-anaesthesia care unit and the ward the PONV protocol consisted of rescue-treatment with an antiemetic drug: either one of the above antiemetic drugs if not previously administered, or metoclopramide 20 mg i.v. There was no scheduled PONV prophylaxis prescribed for patients returning to the ward.
Intervention group
The prediction model was implemented as a directive decision support tool in the anaesthesia information management system (Vierkleurenpen®), a custom-made system written by one of the authors (L.v.W.). The model presented a patient's predicted PONV risk accompanied with advice on the number of prophylactic antiemetics to administer based on that individual's risk (i.e. a directive risk-based approach). The anaesthesia information management system automatically presented this risk and the recommendation to the responsible anaesthetist on the computer screen during each anaesthetic case in each operating theatre. The on-screen presentation was designed as a ‘traffic light’ with four colours (from green, through yellow and orange, to red). The initial colour of the traffic light depended on the patients predicted PONV risk and corresponded to the number of prophylactic antiemetics advised: from zero antiemetics (green) to three antiemetics (red). Anaesthetists then decided whether to follow the advice and administer prophylactic antiemetics accordingly. The colour—and hence the advice—did not change until an antiemetic drug was administered. The software ‘adjusted’ the predicted risk by a 26% relative risk reduction per antiemetic, which is the previously reported relative risk reduction.24 When the adjusted risk fell below a threshold of 26% plus 4% for each antiemetic that already had been administered, the traffic light would turn ‘green’ and no further prophylactic antiemetics were advised. The 4% addition per antiemetic was aimed to ease the achievement of a ‘green light’ in high-risk patients, as very high-risk patients otherwise would never get a ‘green light’ even when treated with all available antiemetics. Based on the calculations by the software tool, the predicted PONV risk was classified into one of the four recommendation categories with an initial traffic light colour: no antiemetics (green) <26% predicted risk; 1 antiemetic <41% predicted risk (yellow); 2 antiemetics below 62% predicted risk (orange); and 3 antiemetics for 62% or greater predicted risk (red). Recommendation thresholds were adapted from the 2007 Society of Ambulatory Anesthesia guidelines on the management of PONV.1 The recommendations did not specify which prophylactic antiemetics to give, but it was advised to follow the order of the existing, local protocol (see ‘Care-as-usual group’ section). The local protocol was easily accessible through a direct link on the main screen of the anaesthesia information management system.
Outcome and follow-up
The incidence of PONV was defined as the occurrence of at least one of the following events within the first 24 h after surgery: an episode of nausea, an episode of vomiting, or the administration of any rescue antiemetic. For nausea, the patient was asked to rate their feeling of nausea during the preceding period on a three-point verbal rating scale (no/yes, a bit/yes, and definitely) and for the analysis the variable was dichotomized to any nausea (no/yes). Vomiting was defined as the expulsion of gastric contents and was recorded as a binary outcome (no/yes). Research nurses and trained medical students collected data on the occurrence of postoperative nausea using a validated questionnaire.6,25 Data were collected at the post-anaesthesia care unit (30 min and 60 min after arrival, and when leaving the unit), and 24 h after surgery at the ward. The outcome variable for PONV was coded as missing when any of the follow-up measurements had not been completed.
The administration of risk-dependent PONV prophylaxis (physician behaviour) was defined as the number of prophylactic antiemetics administered per patient and was recorded in the anaesthesia information management system. The use of total i.v. anaesthesia was not counted as a prophylactic intervention regarding the primary outcome, as it was unlikely to change during the anaesthetic case. However, as inhalation anaesthesia was a predictor within the prediction model, the presented recommendation did depend on the type of anaesthesia used.
Statistical analysis
Analysis was performed under the intention-to-treat principle. All statistical analyses were performed in the R software (version 2.15.0). Statistical significance was defined as a two-sided α of 0.05. Continuous variables were visually assessed for a normal distribution using histograms and QQ plots. Parametric variables were expressed as means with standard deviations, non-parametric variables were expressed as medians with inter-quartile ranges, and discrete variables were expressed as numbers with percentages.
The crude data on the administration and the incidence of PONV are shown in four risk categories according to the predictions and recommendations made by the decision support tool. Mixed-effects regression analyses were used for both outcomes: logistic regression for the incidence of PONV and linear regression for the number of prophylactic antiemetics per patient (glmer, lme4 package, and R software). A random intercept was included in the models, as the study was clustered by anaesthetists. For both outcomes, allocation group, predicted PONV risks, and interaction between allocation group and predicted PONV risk were included as independent variables in the model. As the primary interest was a risk-dependent effect, we included the interaction term between the allocation group and predicted risk. The interaction term quantified the difference in treatment effect (between intervention and care-as-usual) across predicted risks (e.g. an OR <1 would signify that a reduction in PONV because of the directive approach was greater in patients with higher risks). For the PONV incidence analysis predicted PONV risks were included as a continuous variable (i.e. the PONV risk on the log odds scale). For the analysis of physician behaviour, the relation between the intervention (the recommended number of antiemetics per predicted PONV risk category) and the outcome (the actual number of administered prophylactic antiemetics) was hypothetically expected to be linear (0 recommended antiemetics was likely to result in 0 administered antiemetics, 1 recommended antiemetic was likely to result in 1 administered antiemetic, etc.). Therefore, not the continuous predicted risk variable but rather the advised number of antiemetics and its interaction with allocation group were used as independent variables in a linear regression model with the administered number of antiemetics as the dependent variable.
As this was a non-randomized study, we had to adjust for potential differences between the care-as-usual and intervention group. Although inclusion of the predicted risk variable and its interaction term with allocation groups would probably adjust for most of the confounding, we a priori decided to additionally adjust for all variables from Table 2 that are either risk factors for PONV or may influence the decision on PONV prophylaxis (e.g. ASA class).26 Furthermore, the anaesthetists may have lowered their opioid prescription rates in patients with a high predicted risk to prevent PONV. As opioid usage was an intermediate variable, we performed a separate sensitivity analysis to adjust the primary outcome analysis for opioid usage.
Patient characteristics. Age, predicted risk of PONV are presented as mean (sd), operation duration as median (first quartile, third quartile). Other values are absolute frequencies (percentages). *N represents the total number of non-missing observations for that characteristic. PONV, postoperative nausea and vomiting
| N* | Care-as-usual group (n=1022), mean (sd) | Intervention group (n=458), mean (sd) | |
|---|---|---|---|
| Age, yr, mean (range) | 1480 | 52 (18–94) | 54 (18–88) |
| Female gender | 1480 | 496 (49) | 193 (42) |
| ASA class | 1477 | ||
| I | 337 (33) | 121 (27) | |
| II | 561 (55) | 272 (60) | |
| III | 116 (11) | 62 (14) | |
| IV | 8 (1) | 0 (0) | |
| Current smoking | 1453 | 288 (29) | 109 (24) |
| Surgery with a high PONV risk | 1104 | 82 (12) | 53 (13) |
| History of PONV/motion sickness | 1397 | 212 (22) | 66 (15) |
| Inhalation anaesthesia | 1480 | 472 (46) | 224 (49) |
| Predicted risk of PONV | 1480 | 0.40 (0.13) | 0.39 (0.12) |
| Predicted PONV risk in categories | 1480 | ||
| <26% (0 antiemetics advised) | 127 (12) | 60 (13) | |
| 26–41% (1 antiemetic advised) | 443 (43) | 217 (47) | |
| 41–62% (2 antiemetics advised) | 384 (38) | 161 (35) | |
| >62% (3 antiemetics advised) | 68 (7) | 20 (4) | |
| Operation duration, min | 1480 | 128 (85, 188) | 133 (86, 206) |
| N* | Care-as-usual group (n=1022), mean (sd) | Intervention group (n=458), mean (sd) | |
|---|---|---|---|
| Age, yr, mean (range) | 1480 | 52 (18–94) | 54 (18–88) |
| Female gender | 1480 | 496 (49) | 193 (42) |
| ASA class | 1477 | ||
| I | 337 (33) | 121 (27) | |
| II | 561 (55) | 272 (60) | |
| III | 116 (11) | 62 (14) | |
| IV | 8 (1) | 0 (0) | |
| Current smoking | 1453 | 288 (29) | 109 (24) |
| Surgery with a high PONV risk | 1104 | 82 (12) | 53 (13) |
| History of PONV/motion sickness | 1397 | 212 (22) | 66 (15) |
| Inhalation anaesthesia | 1480 | 472 (46) | 224 (49) |
| Predicted risk of PONV | 1480 | 0.40 (0.13) | 0.39 (0.12) |
| Predicted PONV risk in categories | 1480 | ||
| <26% (0 antiemetics advised) | 127 (12) | 60 (13) | |
| 26–41% (1 antiemetic advised) | 443 (43) | 217 (47) | |
| 41–62% (2 antiemetics advised) | 384 (38) | 161 (35) | |
| >62% (3 antiemetics advised) | 68 (7) | 20 (4) | |
| Operation duration, min | 1480 | 128 (85, 188) | 133 (86, 206) |
Patient characteristics. Age, predicted risk of PONV are presented as mean (sd), operation duration as median (first quartile, third quartile). Other values are absolute frequencies (percentages). *N represents the total number of non-missing observations for that characteristic. PONV, postoperative nausea and vomiting
| N* | Care-as-usual group (n=1022), mean (sd) | Intervention group (n=458), mean (sd) | |
|---|---|---|---|
| Age, yr, mean (range) | 1480 | 52 (18–94) | 54 (18–88) |
| Female gender | 1480 | 496 (49) | 193 (42) |
| ASA class | 1477 | ||
| I | 337 (33) | 121 (27) | |
| II | 561 (55) | 272 (60) | |
| III | 116 (11) | 62 (14) | |
| IV | 8 (1) | 0 (0) | |
| Current smoking | 1453 | 288 (29) | 109 (24) |
| Surgery with a high PONV risk | 1104 | 82 (12) | 53 (13) |
| History of PONV/motion sickness | 1397 | 212 (22) | 66 (15) |
| Inhalation anaesthesia | 1480 | 472 (46) | 224 (49) |
| Predicted risk of PONV | 1480 | 0.40 (0.13) | 0.39 (0.12) |
| Predicted PONV risk in categories | 1480 | ||
| <26% (0 antiemetics advised) | 127 (12) | 60 (13) | |
| 26–41% (1 antiemetic advised) | 443 (43) | 217 (47) | |
| 41–62% (2 antiemetics advised) | 384 (38) | 161 (35) | |
| >62% (3 antiemetics advised) | 68 (7) | 20 (4) | |
| Operation duration, min | 1480 | 128 (85, 188) | 133 (86, 206) |
| N* | Care-as-usual group (n=1022), mean (sd) | Intervention group (n=458), mean (sd) | |
|---|---|---|---|
| Age, yr, mean (range) | 1480 | 52 (18–94) | 54 (18–88) |
| Female gender | 1480 | 496 (49) | 193 (42) |
| ASA class | 1477 | ||
| I | 337 (33) | 121 (27) | |
| II | 561 (55) | 272 (60) | |
| III | 116 (11) | 62 (14) | |
| IV | 8 (1) | 0 (0) | |
| Current smoking | 1453 | 288 (29) | 109 (24) |
| Surgery with a high PONV risk | 1104 | 82 (12) | 53 (13) |
| History of PONV/motion sickness | 1397 | 212 (22) | 66 (15) |
| Inhalation anaesthesia | 1480 | 472 (46) | 224 (49) |
| Predicted risk of PONV | 1480 | 0.40 (0.13) | 0.39 (0.12) |
| Predicted PONV risk in categories | 1480 | ||
| <26% (0 antiemetics advised) | 127 (12) | 60 (13) | |
| 26–41% (1 antiemetic advised) | 443 (43) | 217 (47) | |
| 41–62% (2 antiemetics advised) | 384 (38) | 161 (35) | |
| >62% (3 antiemetics advised) | 68 (7) | 20 (4) | |
| Operation duration, min | 1480 | 128 (85, 188) | 133 (86, 206) |
Before multivariable modelling, all continuous variables were tested for non-linearity using restricted cubic splines.27 Missing data were multiply imputed (n=10) using a regression approach in R (aregImpute, Hmisc package). Imputation of missing variables was based on predictors, outcome variables, and other perioperative data.28–31 As PONV was coded missing when any of the follow-up measurements was incomplete, the non-missing follow-up measurements of PONV were added to the imputation process to serve as auxiliary variables to impute missing values for PONV. Subsequently, the imputed values for PONV were included into the mixed-effects regression models, instead of deleted.32 The anaesthetist identity was added as an extra variable to the imputation model to take into account clustering in the data.
Results
A total of 1480 patients were included in the study: 1022 during the care-as-usual period and 458 during the intervention period. The mean predicted PONV risks were comparable between allocation periods. However, small differences in the distribution of predicted PONV risk categories existed between allocation periods. Differences in baseline of several predictor variables are likely to be related to the small differences in the distribution of the predicted PONV risk (Table 2).
In total, 81% of all the follow-up measurements on PONV were completed (intervention group 75%; care-as-usual group 83%), with 92% of all patients having at least one follow-up measurement completed (intervention period 87%; care-as-usual period 94%). The incidence of PONV was 42% during the intervention period, compared with 50% during the care-as-usual period. Table 3 (left panel) shows the crude, risk-dependent effect of the intervention on the incidence of PONV. Confirmed by the regression analysis, there was a significant reduction in the incidence of PONV during the intervention period in comparison with the care-as-usual period (OR: 0.60, 95% CI: 0.43–0.83), with a greater reduction in high-risk patients (OR interaction term: 0.45, 95% CI: 0.28–0.72). The statistical significance of the risk-dependent reduction in PONV and the risk-dependent numbers needed to treat (NNT) are reflected in Figure 1a. Adjustment for baseline characteristics or opioid usage did not change these results and inferences. Results of the adjusted and unadjusted regression analyses can be found in Table 4. Differences in ORs for the variable predicted risk between complete case analysis and after multiple imputations can be explained by the confounder correction using all predictors from the prediction model.
Primary and secondary outcomes per category of predicted risk of PONV. *Data represent absolute numbers (%) of PONV. †Data represent absolute numbers (%) of patients who received either 0, 1, 2, or 3 of the available prophylactic antiemetics: ondansetron, droperidol, dexamethasone, or both. Grey cells represent cells where anaesthetists administered the exact number of administered prophylactic antiemetics as recommended for that risk category (i.e. the compliance). ‡N represents the total number of non-missing observations for that characteristic. PONV, postoperative nausea and vomiting
| Predicted PONV risk in categories | Incidence of PONV* | Number of prophylactic antiemetics† | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | |||||||
| 0 | 1 | 2 | 3 | 0 | 1 | 2 | 3 | |||||
| <26% (0 antiemetics advised) | 116 | 19 (23) | 10 (29) | 188 | 117 (92) | 10 (8) | 0 (0) | 0 (0) | 53 (88) | 7 (12) | 0 (0) | 0 (0) |
| 26–41% (1 antiemetic advised) | 432 | 123 (41) | 50 (38) | 661 | 362 (82) | 69 (16) | 11 (2) | 1 (0) | 66 (30) | 141 (65) | 10 (5) | 0 (0) |
| 41–62% (2 antiemetics advised) | 354 | 167 (64) | 48 (52) | 546 | 258 (67) | 98 (26) | 21 (5) | 7 (2) | 35 (22) | 20 (12) | 103 (64) | 3 (2) |
| >62% (3 antiemetics advised) | 60 | 41 (82) | 5 (50) | 88 | 31 (46) | 28 (41) | 9 (13) | 0 (0) | 3 (15) | 3 (15) | 7 (35) | 7 (35) |
| Predicted PONV risk in categories | Incidence of PONV* | Number of prophylactic antiemetics† | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | |||||||
| 0 | 1 | 2 | 3 | 0 | 1 | 2 | 3 | |||||
| <26% (0 antiemetics advised) | 116 | 19 (23) | 10 (29) | 188 | 117 (92) | 10 (8) | 0 (0) | 0 (0) | 53 (88) | 7 (12) | 0 (0) | 0 (0) |
| 26–41% (1 antiemetic advised) | 432 | 123 (41) | 50 (38) | 661 | 362 (82) | 69 (16) | 11 (2) | 1 (0) | 66 (30) | 141 (65) | 10 (5) | 0 (0) |
| 41–62% (2 antiemetics advised) | 354 | 167 (64) | 48 (52) | 546 | 258 (67) | 98 (26) | 21 (5) | 7 (2) | 35 (22) | 20 (12) | 103 (64) | 3 (2) |
| >62% (3 antiemetics advised) | 60 | 41 (82) | 5 (50) | 88 | 31 (46) | 28 (41) | 9 (13) | 0 (0) | 3 (15) | 3 (15) | 7 (35) | 7 (35) |
Primary and secondary outcomes per category of predicted risk of PONV. *Data represent absolute numbers (%) of PONV. †Data represent absolute numbers (%) of patients who received either 0, 1, 2, or 3 of the available prophylactic antiemetics: ondansetron, droperidol, dexamethasone, or both. Grey cells represent cells where anaesthetists administered the exact number of administered prophylactic antiemetics as recommended for that risk category (i.e. the compliance). ‡N represents the total number of non-missing observations for that characteristic. PONV, postoperative nausea and vomiting
| Predicted PONV risk in categories | Incidence of PONV* | Number of prophylactic antiemetics† | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | |||||||
| 0 | 1 | 2 | 3 | 0 | 1 | 2 | 3 | |||||
| <26% (0 antiemetics advised) | 116 | 19 (23) | 10 (29) | 188 | 117 (92) | 10 (8) | 0 (0) | 0 (0) | 53 (88) | 7 (12) | 0 (0) | 0 (0) |
| 26–41% (1 antiemetic advised) | 432 | 123 (41) | 50 (38) | 661 | 362 (82) | 69 (16) | 11 (2) | 1 (0) | 66 (30) | 141 (65) | 10 (5) | 0 (0) |
| 41–62% (2 antiemetics advised) | 354 | 167 (64) | 48 (52) | 546 | 258 (67) | 98 (26) | 21 (5) | 7 (2) | 35 (22) | 20 (12) | 103 (64) | 3 (2) |
| >62% (3 antiemetics advised) | 60 | 41 (82) | 5 (50) | 88 | 31 (46) | 28 (41) | 9 (13) | 0 (0) | 3 (15) | 3 (15) | 7 (35) | 7 (35) |
| Predicted PONV risk in categories | Incidence of PONV* | Number of prophylactic antiemetics† | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | N‡ | Care-as-usual group (n=1022) | Intervention group (n=458) | |||||||
| 0 | 1 | 2 | 3 | 0 | 1 | 2 | 3 | |||||
| <26% (0 antiemetics advised) | 116 | 19 (23) | 10 (29) | 188 | 117 (92) | 10 (8) | 0 (0) | 0 (0) | 53 (88) | 7 (12) | 0 (0) | 0 (0) |
| 26–41% (1 antiemetic advised) | 432 | 123 (41) | 50 (38) | 661 | 362 (82) | 69 (16) | 11 (2) | 1 (0) | 66 (30) | 141 (65) | 10 (5) | 0 (0) |
| 41–62% (2 antiemetics advised) | 354 | 167 (64) | 48 (52) | 546 | 258 (67) | 98 (26) | 21 (5) | 7 (2) | 35 (22) | 20 (12) | 103 (64) | 3 (2) |
| >62% (3 antiemetics advised) | 60 | 41 (82) | 5 (50) | 88 | 31 (46) | 28 (41) | 9 (13) | 0 (0) | 3 (15) | 3 (15) | 7 (35) | 7 (35) |
Logistic regression analysis on the risk-dependent incidence of PONV. Numbers represent ORs with 95% CIs. Bold numbers are statistically significant ORs. *Cases with missing variables were discarded. †Adjusted for possible confounders: age (continuous, restricted cubic splines, five knots), gender, ASA class, current smoking, middle ear of abdominal surgery, history of PONV/motion sickness, use of inhalation anaesthesia, procedure duration (continuous, restricted cubic splines, five knots). ‡The predicted PONV risk on the log odds scale. §Using 10 imputation datasets. PONV, postoperative nausea and vomiting; pred., predicted
| Complete case* | |||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.52 (0.35–0.78) | 0.43 (0.27–0.69) | 0.54 (0.36–0.81) |
| Predicted risk‡ | 3.9 (2.9–5.4) | 0.12 (0.0–7.4) | 4.0 (2.9–5.6) |
| Interaction: intervention period* pred. risk‡ | 0.43 (0.24–0.77) | 0.36 (0.18–0.71) | 0.39 (0.21–0.71) |
| Multiple imputation§ | |||
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.64 (0.47–0.89) | 0.60 (0.43–0.83) | 0.66 (0.48–0.90) |
| Predicted risk‡ | 2.9 (2.2–3.8) | 0.73 (0.13–4.2) | 2.9 (2.2–3.8) |
| Interaction: intervention period* pred. risk‡ | 0.49 (0.31–0.77) | 0.45 (0.28–0.72) | 0.47 (0.30–0.73) |
| Complete case* | |||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.52 (0.35–0.78) | 0.43 (0.27–0.69) | 0.54 (0.36–0.81) |
| Predicted risk‡ | 3.9 (2.9–5.4) | 0.12 (0.0–7.4) | 4.0 (2.9–5.6) |
| Interaction: intervention period* pred. risk‡ | 0.43 (0.24–0.77) | 0.36 (0.18–0.71) | 0.39 (0.21–0.71) |
| Multiple imputation§ | |||
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.64 (0.47–0.89) | 0.60 (0.43–0.83) | 0.66 (0.48–0.90) |
| Predicted risk‡ | 2.9 (2.2–3.8) | 0.73 (0.13–4.2) | 2.9 (2.2–3.8) |
| Interaction: intervention period* pred. risk‡ | 0.49 (0.31–0.77) | 0.45 (0.28–0.72) | 0.47 (0.30–0.73) |
Logistic regression analysis on the risk-dependent incidence of PONV. Numbers represent ORs with 95% CIs. Bold numbers are statistically significant ORs. *Cases with missing variables were discarded. †Adjusted for possible confounders: age (continuous, restricted cubic splines, five knots), gender, ASA class, current smoking, middle ear of abdominal surgery, history of PONV/motion sickness, use of inhalation anaesthesia, procedure duration (continuous, restricted cubic splines, five knots). ‡The predicted PONV risk on the log odds scale. §Using 10 imputation datasets. PONV, postoperative nausea and vomiting; pred., predicted
| Complete case* | |||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.52 (0.35–0.78) | 0.43 (0.27–0.69) | 0.54 (0.36–0.81) |
| Predicted risk‡ | 3.9 (2.9–5.4) | 0.12 (0.0–7.4) | 4.0 (2.9–5.6) |
| Interaction: intervention period* pred. risk‡ | 0.43 (0.24–0.77) | 0.36 (0.18–0.71) | 0.39 (0.21–0.71) |
| Multiple imputation§ | |||
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.64 (0.47–0.89) | 0.60 (0.43–0.83) | 0.66 (0.48–0.90) |
| Predicted risk‡ | 2.9 (2.2–3.8) | 0.73 (0.13–4.2) | 2.9 (2.2–3.8) |
| Interaction: intervention period* pred. risk‡ | 0.49 (0.31–0.77) | 0.45 (0.28–0.72) | 0.47 (0.30–0.73) |
| Complete case* | |||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.52 (0.35–0.78) | 0.43 (0.27–0.69) | 0.54 (0.36–0.81) |
| Predicted risk‡ | 3.9 (2.9–5.4) | 0.12 (0.0–7.4) | 4.0 (2.9–5.6) |
| Interaction: intervention period* pred. risk‡ | 0.43 (0.24–0.77) | 0.36 (0.18–0.71) | 0.39 (0.21–0.71) |
| Multiple imputation§ | |||
| Unadjusted | Confounder adjusted† | Adjusted for opioid usage | |
| Intervention period | 0.64 (0.47–0.89) | 0.60 (0.43–0.83) | 0.66 (0.48–0.90) |
| Predicted risk‡ | 2.9 (2.2–3.8) | 0.73 (0.13–4.2) | 2.9 (2.2–3.8) |
| Interaction: intervention period* pred. risk‡ | 0.49 (0.31–0.77) | 0.45 (0.28–0.72) | 0.47 (0.30–0.73) |
Graphical representation of the predicted incidence of PONV (a), and subsequent administration of prophylactic antiemetics by anaesthetists (b). The solid lines and their 95% CIs represent the predicted risk of PONV. The dotted vertical line shows the intersection point of both groups. The mixed-effects models included fixed effects for the variables: allocation group, predicted risk of PONV, interaction between allocation group and predicted risk, and time. Random effects were included for the intercept, predicted risk, interaction between the allocation group and predicted risk, and time. The lines were calculated using the average for the variable time. The 95% CIs were calculated from the covariance matrix for the variables allocation group and its interaction term with predicted risk. The results are considered statistically significant when the 95% CIs of one study group do not overlap with the solid line of the other study group. (a) The occurrence of PONV after receiving prophylaxis, in patients with a particular predicted risk within each group. The differences between the pale green and blue areas represent the effect of implementation of the prediction model on the occurrence of PONV in patients with a particular predicted risk. For each risk increment of 10%, the NNT for PONV within 24 h after surgery was calculated by taking the multiplicative inverse of the absolute risk reduction between the intervention group and the care-as-usual group. When the absolute risk reduction was negative, the number needed to harm was calculated instead. (b) The number of prophylactic antiemetics a patient with a particular predicted risk of PONV would receive from any anaesthetist within each group. The differences between the pale green and blue areas therefore represent the changes in physician behaviour concerning prescription of antiemetic prophylaxis, caused by implementation of the prediction model.
The number and type of prophylactic antiemetics were documented for all patients. During the intervention period, anaesthetists complied with the recommendation of the clinical decision support tool and administered the recommended number of prophylactic antiemetics in 66% of patients. Although no actual recommendations were given during the care-as-usual period, the fictional compliance (i.e. the prescription behaviour that would be recommended, if the decision rule had been active) was 20%, resulting in an absolute increase of compliance of 46%. During the intervention period, 76% of the prophylactic antiemetics were administered at the appropriate time during the anaesthetic case, compared with 72% during the care-as-usual period. The timing of the prophylactic antiemetics was unrelated to the occurrence of PONV, either at the post-anaesthesia care unit or at the ward. The crude, risk-dependent effects of the intervention on administration of prophylactic antiemetics and the risk-dependent compliance are given in Table 3, right panel. The increase in administration of prophylactic antiemetics was confirmed in the results of the linear regression analysis after multiple imputation and confounder adjustment. Anaesthetists administered more antiemetic prophylaxis in a risk-dependent manner during the intervention period. In this period, for each additional antiemetic advised the anaesthetists actually administered 0.49 (95% CI: 0.41–0.58) additional antiemetics (Table 5, regression coefficient for the interaction term, far right column). The statistical significance of the increased administration of risk-dependent PONV prophylaxis is reflected by Figure 1b. These results were not different from the models without adjustment for confounding, as given in Table 5.
Linear regression analysis on physicians’ administration of risk-dependent antiemetic prophylaxis. Numbers represent regression coefficients with 95% CIs. Bold numbers are statistically significant regression coefficients. No unadjusted model is presented for multiple imputation as variables within the models were not missing. *Cases with missing variables were discarded. †Adjusted for possible confounders: age (continuous, restricted cubic splines, five knots), gender, ASA class, current smoking, middle ear of abdominal surgery, history of PONV/motion sickness, use of inhalation anaesthesia, procedure duration (continuous, restricted cubic splines, five knots). ‡Using 10 imputation datasets. §Regression coefficients represent an increase in the number of administered prophylactic antiemetics per advised prophylactic antiemetics (i.e. per risk category). PONV, postoperative nausea and vomiting; pred., predicted
| Complete case* | Multiple imputation‡ | ||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Confounder adjusted† | |
| Intervention period | 0.04 (−0.09–0.17) | 0.06 (−0.10–0.22) | 0.04 (−0.09–0.17) |
| Recommendation categories§ | 0.18 (0.14–0.23) | 0.07 (−0.05–0.20) | 0.04 (−0.05–0.14) |
| Interaction: intervention period* categories§ | 0.48 (0.39–0.56) | 0.49 (0.39–0.58) | 0.50 (0.41–0.58) |
| Complete case* | Multiple imputation‡ | ||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Confounder adjusted† | |
| Intervention period | 0.04 (−0.09–0.17) | 0.06 (−0.10–0.22) | 0.04 (−0.09–0.17) |
| Recommendation categories§ | 0.18 (0.14–0.23) | 0.07 (−0.05–0.20) | 0.04 (−0.05–0.14) |
| Interaction: intervention period* categories§ | 0.48 (0.39–0.56) | 0.49 (0.39–0.58) | 0.50 (0.41–0.58) |
Linear regression analysis on physicians’ administration of risk-dependent antiemetic prophylaxis. Numbers represent regression coefficients with 95% CIs. Bold numbers are statistically significant regression coefficients. No unadjusted model is presented for multiple imputation as variables within the models were not missing. *Cases with missing variables were discarded. †Adjusted for possible confounders: age (continuous, restricted cubic splines, five knots), gender, ASA class, current smoking, middle ear of abdominal surgery, history of PONV/motion sickness, use of inhalation anaesthesia, procedure duration (continuous, restricted cubic splines, five knots). ‡Using 10 imputation datasets. §Regression coefficients represent an increase in the number of administered prophylactic antiemetics per advised prophylactic antiemetics (i.e. per risk category). PONV, postoperative nausea and vomiting; pred., predicted
| Complete case* | Multiple imputation‡ | ||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Confounder adjusted† | |
| Intervention period | 0.04 (−0.09–0.17) | 0.06 (−0.10–0.22) | 0.04 (−0.09–0.17) |
| Recommendation categories§ | 0.18 (0.14–0.23) | 0.07 (−0.05–0.20) | 0.04 (−0.05–0.14) |
| Interaction: intervention period* categories§ | 0.48 (0.39–0.56) | 0.49 (0.39–0.58) | 0.50 (0.41–0.58) |
| Complete case* | Multiple imputation‡ | ||
|---|---|---|---|
| Unadjusted | Confounder adjusted† | Confounder adjusted† | |
| Intervention period | 0.04 (−0.09–0.17) | 0.06 (−0.10–0.22) | 0.04 (−0.09–0.17) |
| Recommendation categories§ | 0.18 (0.14–0.23) | 0.07 (−0.05–0.20) | 0.04 (−0.05–0.14) |
| Interaction: intervention period* categories§ | 0.48 (0.39–0.56) | 0.49 (0.39–0.58) | 0.50 (0.41–0.58) |
Discussion
We studied the effects of the implementation of risk-dependent PONV prophylaxis based on the predicted PONV risks, calculated by a prediction model. The model provided automated decision support by presenting predicted risks directly accompanied with treatment recommendations to anaesthetists in the operating theatre (i.e. a directive approach was used).12–14,20 This directive approach clearly increased administration of risk-dependent antiemetic prophylaxis to patients and also reduced the incidence of PONV within 24 h after surgery, particularly in patients at higher risk of PONV.
The results of the present study are in contrast with the results of our previous study. The previous study—a large cluster-randomized trial—tested an assistive approach for model implementation; that is, presenting only the risk of PONV without a therapeutic recommendation. This assistive strategy had little effect on the PONV incidence, whereas the directive strategy of the present study significantly reduced (OR: 0.60, 95% CI: 0.43–0.83) the incidence of PONV within 24 h after surgery.17 The difference in results between the two studies suggests that the impact on clinical practice may be larger when a prediction model is accompanied with an actionable recommendation to aid physicians in their decision-making.
The impact on physician behaviour in the present study is similar to other impact studies of PONV decision support.7,8 Kooij and colleagues implemented a directive decision support tool with an absolute increase in compliance of 40%, which is comparable with the results of our present (directive) study (46%). Using an assistive approach Frenzel and colleagues achieved an absolute increase in compliance of 5%, with a single feedback report, which is similar to our previous (assistive) study.17 Effects on the incidence of PONV (absolute risk reduction of 8%) for our directive approach were within the lower range of results from other studies that reported overall absolute risk reductions ranging from 8 to 35%.16,33–37 Unfortunately, the merit of such a comparison is limited, attributable to differences in actual administration of PONV prophylaxis, study design and study analysis. Most of the other studies did not randomize, did not adjust for confounding, or did not have a proper control group, which makes it difficult to compare their results to our study.
Regardless of a significant decrease in PONV incidence, the actual impact of the directive approach on PONV occurrence seems at best moderate and does not come close to its desired potential impact—a ‘PONV-free hospital’.38 However, differences between actual and potential impact do not imply that we should discard risk-dependent strategies for administration of PONV prophylaxis. Several interactions between clinicians and the decision support tool need to be considered for our study, before coming to a conclusion.12
First, the predictive performance of the prediction model may have been insufficient to improve clinical decision-making. The predictive performance of our prediction model was comparable with other PONV prediction models (c-statistic ∼0.70).4,6,23 With a moderate predictive performance, decisions based on the model may not have been superior to care-as-usual (i.e. clinical judgement). Secondly, the interface of the decision support tool may have affected the compliance to the recommendations. For example, desensitization may have occurred, as the colour would often not change during the surgical part of the case, as prophylactic antiemetics are supposed to be administered either at the start or towards the end of the anaesthetic case. Thirdly, despite a large increase in risk-dependent PONV prophylaxis, physicians did not fully adhere to the presented risks and therapeutic recommendations. For example, patients in the highest risk category where three prophylactic antiemetics were advised, received on average two prophylactic antiemetics. Several barriers to use prediction models and decision support have been identified in the literature and may account for the incomplete compliance by the anaesthetists.39–42 Fourthly, the increased administration of prophylactic antiemetics may have been an effect of an overall increase in attention to PONV (a Hawthorne effect), instead of an effect of the intervention. It is the actual goal of decision support to increase the attention for a particular patient problem, hence decision support systems are sometimes referred to as ‘reminder systems’.43 In the case of a prediction model as a form of decision support, the goal is to actually improve risk-dependent decision-making. As there was a risk-dependent effect on both primary and secondary outcome in our study, it is likely that the anaesthetists indeed used the information presented by the decision support tool, instead of a Hawthorne effect.
The results of our study provide some support for a general multimodal prevention strategy presented in the recent update of the consensus guidelines for management of PONV from the Society of Ambulatory Anesthesia (SAMBA).2 In a general multimodal prevention strategy prophylactic antiemetics are routinely administered to all patients, which implies that more prophylactic antiemetics will be administered to patients with low-to-moderate risks of PONV. The risk thresholds and treatment recommendations as used in our decision support tool were developed from clinical considerations, as literature on specific treatment recommendations for our prediction model was not available at the time of the study's initiation.1 From the NNT in Figure 1 we can observe that our prevention strategy was mainly effective in patients at highest risk of PONV, for which the NNT dropped <10 for patients with a predicted risk per cent of ≥50%. If anaesthetists should adopt a more liberal approach to administration of prophylactic antiemetics, our decision support tool would recommend more antiemetics to patients at lower risk. This may improve the therapeutic efficiency in patients with predicted risks <50%, resulting in a PONV incidence that may be closer to a ‘PONV-free hospital’ than the resultant PONV incidence in our study.
There is an ongoing debate as to whether a more liberal use of prophylactic antiemetics should be adopted by either a risk-dependent strategy or routine administration of several prophylactic antiemetics to every patient (a general multimodal approach).2,9–11,36 At this time, neither points of view have a substantive evidence base. On the one hand, there is a debate on the clinical utility of prediction models, as their implementation rate remains very low. Moreover, a general multimodal approach would simplify PONV prevention as there is no need for reminders. Decision support tools may then be reserved for more complicated clinical decisions, which require interpretation of more sophisticated—and non-routine—information. On the other hand, antiemetics have side-effects and there is some reluctance to administer polypharmacy with several, potentially unnecessary drugs, and a prediction model may assist in achieving an optimal risk-to-benefit ratio. This study adds some support for both points of view. It is one of the first prospective comparative studies that demonstrate that risk-dependent prophylaxis actually decreases the incidence of PONV. At the same time, anaesthetists were only partially compliant with the presented recommendations and the absolute decrease in PONV incidence was limited. If the anaesthesia community is willing to embrace large-scale multimodal prevention of PONV—as suggested by the recently updated SAMBA guidelines on PONV management—this may enable us to evaluate whether a ‘PONV-free hospital’ is actually achievable with an acceptable rate of side-effects.
We conclude that risk-dependent PONV prophylaxis is not only efficacious in clinical trials, but also effective in clinical practice when a real-time, computer-based prediction model is used in combination with risk-based recommendations on PONV prophylaxis. Implementation of a risk prediction model combined with treatment recommendations per predicted risk, yields better effects on clinical decision-making and patient outcomes than the use of a prediction model without addition of such treatment recommendations. As the resultant PONV incidence remained high, a more liberal use of prophylactic antiemetics and lowering of the risk thresholds for the actionable recommendations may be needed to achieve a truly ‘PONV-free hospital’.
Authors' contributions
T.H.K.: study concept and design, acquisition of data, statistical analysis, interpretation of data, drafting of the manuscript, and critical revision of the manuscript for important intellectual content; Y.V.: study concept and design, statistical analysis, interpretation of data, drafting of the manuscript, and critical revision of the manuscript for important intellectual content; L.v.W.: study concept and design, acquisition of data, interpretation of data, and critical revision of the manuscript for important intellectual content; C.J.K.: study concept and design, interpretation of data, and critical revision of the manuscript for important intellectual content; K.G.M.M.: study concept and design, statistical analysis, interpretation of data, and critical revision of the manuscript for important intellectual content; W.A.v.K.: study concept and design, interpretation of data, drafting of the manuscript, and critical revision of the manuscript for important intellectual content.
Funding
This work was supported by The Netherlands Organisation for Health Research and Development (ZonMW), The Hague, South Holland, The Netherlands. ZonMW project numbers: 945.16.202; 912.08.004; 918.10.615. The funding agreement ensured the authors' independence in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Declaration of interest
None declared.
References
Author notes
.
