Advancing electronic health record vendor usability maturity: Progress and next steps

Abstract

Despite basic federal requirements promoting a user-centered design approach to electronic health record (EHR) development and usability testing there have been usability and safety risks with EHR technology. Four EHR vendors were asked to provide written descriptions of their usability practices, and we reviewed these descriptions to identify areas where there has been advancement and areas for improvement. All 4 vendors described user-centered design processes and usability testing methods that demonstrate advancement from previous studies of vendor practices. Importantly, vendors are also beginning to address aspects of EHR implementation that play a critical role in shaping EHR usability. There are important areas for improvement in vendor practices including a greater focus on safety and on measurement and benchmarking. Vendors sharing their current usability practices demonstrates a step toward greater transparency which has typically been lacking.

Electronic health record (EHR) usability, the extent to which this technology can be used efficiently, effectively, satisfactorily, and safely has been a source of clinician and patient frustration, and poor usability poses patient safety risks.^1,2 Despite the Office of the National Coordinator for Health Information Technology having basic user-centered design (UCD) and usability testing requirements as part of their health information technology (IT) certification program, usability has been suboptimal.^3,4 Recent research examining system usability scale (SUS) scores, a common measure of product usability, collected from physicians using EHRs in clinical practice equated to an “F” grade.⁵ Further, research shows that there was no significant increase in SUS scores for many EHR vendors over time as reported in certification documents.⁶

Vendor user-centered design (UCD) and usability evaluation processes during design and development are critical factors that shape the ultimate, implemented product’s usability in actual clinical practice. All 4 vendors that provided descriptions of their usability frameworks described a focus on end users of the product, the importance of conducting observations and using ethnographic methods to better understand the context of use, and an iterative development process that incorporates user feedback throughout product development.⁷ The descriptions suggest in-depth UCD knowledge. Some of the vendors have also begun to recognize the different sociotechnical system factors (including the personnel, hardware, software, workflow, communication, and culture of the organization) that shape EHR usability.⁸ Contrasting these frameworks with research from 2015 into the usability practices of 11 EHR vendors, which found that most of those vendors had little to no knowledge of UCD with few or no usability experts employed by the company, the recent descriptions by these 4 vendors demonstrate progressively advancing usability maturity, which is promising.^9,10

However, vendor UCD and usability evaluation processes are not the only factors that shape EHR usability. Implementation processes such as healthcare facility configuration and customization choices, training, and policy all impact usability of EHR technology. The difference between EHR products pre- and postimplementation, called the EHR usability reality gap, continues to be a significant challenge.¹¹ The Office of the National Coordinator for Health Information Technology’s basic usability-focused certification requirements apply to the product as developed by the EHR vendor and the usability testing conducted by the vendor is generally done in a controlled, simulated environment.¹² When an EHR product is implemented, it is generally configured and customized such that results of any usability testing and safety assessments prior to implementation may no longer be applicable. Further, during real-world use, the user’s performance can be affected by multiple sociotechnical system factors. Because most healthcare facilities do not have enough or appropriately trained workforce to optimally configure and usability or safety test their product, frontline clinicians interact with a system that may be vastly different from what was originally tested by the vendor, giving rise to the usability reality gap.¹¹

Three of the 4 vendor descriptions provide insights on how these vendors are working to fill the EHR usability reality gap.⁷ Vendor B provides their customers with a usability testing toolkit and training on usability practices including evidence-based user modeling approaches. Vendor C describes a greater focus on evaluation of implemented products and providing healthcare facilities with greater expertise to optimize product configurations. To achieve this greater focus, vendor C used several approaches, including the use of a third party human factors research firm for an impartial perspective as well as encouraging formative usability testing with the ability for teams to postpone deadlines if a usability problem needed to be addressed. In addition, vendor C noted the use of deploying user experience professionals during the postimplementation period and providing “best choices” and guidance on customization and in some cases limiting customization options to reduce the chances for a less than optimal configuration. Finally, the vendor also implemented formal UCD training classes for their associates and clients to ensure a consistent design and implementation approach. Vendor D describes a usability model that extends into implementation and considers the entire product lifecycle. Specifically, the team incorporates formative usability testing with representative end users during the development process as well as during the postdeployment period. The use of real-world workflows, an iterative design process with clinicians and longitudinal analysis of functions allows the team to take a UCD approach throughout the product lifecycle. These types of practices are critically important steps to improving EHR usability and should be adopted by all EHR vendors. Furthermore, healthcare systems and practices implementing EHRs must consider and address specific, local practice variations that may require further UCD and customization to adequately meet users’ needs within their specific context.

Across the 4 vendor descriptions, there are areas for continued focus and improvement.⁷ There has been extensive research demonstrating the impact of usability on patient safety, yet only 1 vendor mentioned identifying safety issues during the usability process, and this was not a central focus. EHR vendors, in partnership with healthcare facilities, should prioritize patient safety and usability processes, and evaluations should be conducted frequently with both objective and subjective indicators of product safety. Real-time reporting of usability issues should be made available, accessible, and easy to use, and inform actionable improvements. Any aspects of the product that may pose safety risks should be immediately assessed for both the severity and likelihood of harming patients and addressed accordingly.¹³ The other area for improvement is usability measurement and benchmarking. Ensuring that rigorous methods for measuring efficiency, effectiveness, satisfaction, and safety, beyond the SUS, is important to improving EHR usability across all products. For example, use of rigorous standardized test case scenarios across vendors would provide the type of measurement needed to determine whether usability is improving and would provide greater transparency on product usability. Specifically, sharing the safety design issues across vendors could reduce the chances of vendors implementing similar unsafe designs across products. Finally, some of the 4 vendors described platforms to collect customer feedback on their products and ways for customers to suggest new features. While this is important in terms of meeting customer needs from a business perspective, this should not be viewed as a substitute for user-centered and human factors methods that focus on analyzing how clinicians do their work and how products must be designed to meet these needs. Often, both expert and novice end users cannot explain how they do their work, may misrepresent their needs, and may not be able to imagine how a product can be designed and developed to meet their needs.¹⁴

We would like to complement the participating vendors for taking a bold step forward in EHR usability by providing descriptions of their practices, which are typically protected by vendors.⁷ We recognize that the improved UCD and usability evaluation practices described by the vendors will take time to propagate to the products used by frontline clinicians given design and development timelines, certification requirements, and upgrade schedules. Vendor sharing of their maturing usability practices provides hope to the broader community by showing their desire to meet the needs of frontline clinicians and patients. Moving forward, it will be important to continue to have usability as a top priority, and to ensure that safety becomes a top priority, for all stakeholders, including vendors, healthcare facilities, researchers, policymakers, and patients who all have shared responsibility for effective EHR technology.

FUNDING

This work was supported by the Agency for Healthcare Research and Quality grant number R01 HS025136 (to RMR and AZH).

AUTHOR CONTRIBUTIONS

All authors contributed sufficiently and meaningfully to the conception, design, drafting, editing, and revising the article. All authors approved the final version for submission and agree to be accountable for all aspects of the work.

DATA AVAILABILITY STATEMENT

The data underlying this article will be published in the appendix for the issue's editorial

ACKNOWLEDGMENTS

The opinions expressed in this document are those of the authors and do not necessarily reflect the official position of the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

CONFLICT OF INTEREST STATEMENT

The authors have no competing interests to declare.

REFERENCES