AI can help to help refine and target the myriad computerised alerts intended to assist doctors and other team members in day-to-day clinical decision-making, accordoing to a new study from the US.
These pop-up notifications advise individuals on anything from drug contraindications to gaps in patient care documentation.
But exclusion criteria and targeting of these alerts is wanting and up to 90 per cent are ignored, contributing to “alert fatigue.”
From an information technology perspective, throwing human experts at the targeting problem looks slow, expensive and somewhat hit-and-miss.
Lead author Siru Liu, PhD, assistant professor of Biomedical Informatics at anderbilt University Medical Center (VUMC) said: “Across health care, most of these well-intentioned automated alerts are overridden by busy users.
“The alerts serve an essential purpose, but the need to improve them is clear to everyone.”
Liu, senior author Adam Wright, PhD, professor of Biomedical Informatics and director of the Vanderbilt Clinical Informatics Center, and colleagues reported the study in the Journal of the American Medical Informatics Association.
Liu developed a machine learning approach to analyse two years of data on user interactions with alerts at VUMC.
Based on patient characteristics, the model accurately predicted when specific alerts would be dismissed by users.
The researcher then used various processes and methods to peer inside the predictive model, understand its reasoning, and generate suggested improvements to alert logic.
This step, termed explainable AI, or AIX, involved transforming the model’s predictions into rules explaining when users are less likely to accept alerts.
For example, “If the patient is a hospice patient, then the user is less likely to accept the breast cancer screening alert.”
Out of 1,727 suggestions analysed, 76 were found to match later manual updates to VUMC alerts and another 20 were found to align with best practices as determined through interviews with clinicians.
The authors calculated that these 96 recommendations would have eliminated 9.3 per cent of the nearly 3 million alerts analysed in the study, cutting disruptive pop-ups while maintaining patient safety.
“The alignment of the model’s suggestions with manual adjustments made by clinicians to alert logic underscores the robust potential of this technology to enhance health care quality and efficiency,” Liu said.
“Our approach can identify areas overlooked in manual reviews and transform alert improvement into a continuous learning process.”
Beyond refining alerts, Liu added, the methodology uncovered situations indicating problems in workflow, education or staffing.
In this way the approach might more broadly improve quality.
Liu said: “The transparency of our model unveiled scenarios where alerts are dismissed due to downstream issues beyond the alerts themselves.”
Liu and her colleagues have several related projects under consideration, including a multisite prospective study of the effects on patient care of machine learning for CDS improvement; designing an interface for CDS experts to visualize the AIX process and evaluate model-generated suggestions; and exploring capabilities of large language models like ChatGPT for optimising CDS alerts based on user comments and current research literature.
