Clickbusters letter response

We appreciate the thoughtful letter by Dr. Kannry regarding our paper, “Clinician Collaboration to Improve Clinical Decision Support: The Clickbusters Initiative.”¹ In his letter, Dr. Kannry highlights the distinction between medication decision support (MDS) and clinical decision support (CDS) and asserts that analyses of overrides between the 2 may not be comparable. We acknowledge the difference between the 2 types of CDS, but we respectfully disagree with the size of the gap in override rates. Epic provides median and quartile rates for its organizations across more than 800 metrics for benchmarking, including medication warnings (ie, MDS) and BestPractice Advisories (BPAs, ie, CDS). During May 2023, in the inpatient setting, interruptive medication warnings and BPAs had a median override or nonacceptance rate of 87.05% and 89.05%, respectively, and in the outpatient settings, the rates were 88.64% and 87.56%.²

We wholeheartedly agree with Dr. Kannry’s concern about the lack of standardization for CDS measurement and benchmarking. We have seen, in our own work, how differences in the way that CDS measures are operationalized can lead to large differences in even simple measures like alert firing and acceptance rate. In 1 analysis, we reviewed MDS alerts during a 1-month period across 2 institutions and found that alert firing rates differed by more than 60% when comparing unique alerts and total alerts. Similarly, override rates also differed when considering total override responses (66.5%, 78.7%), initial overrides (62.3%, 77.9%), and overrides where medication orders were not discontinued within 24 h (50.7%, 62.8%).³

We also wish to highlight the Clinical Decision Support Innovation Collaborative (CDSiC), a national initiative supported by Agency for Healthcare Research and Quality, which includes workgroups on scaling, measurement, and dissemination and CDS outcomes and objectives. These workgroups are focusing efforts to compile and recommend approaches for measuring and reporting on both CDS implementations and outcomes in a standardized way, with a focus on patient-centered CDS.⁴ In summary, we agree that more research is needed, and we encourage CDS researchers and practitioners, including informaticians at healthcare organizations and EHR vendors, to collaborate, potentially with the support of AMIA, to work toward better standardization in this important area.

AUTHOR CONTRIBUTIONS

All authors participated in the research and contributed to the writing and final review of the manuscript.

DATA AVAILABILITY

There are no new data associated with this article.

REFERENCES