Mahajan, Prashant

Objectives: Electronic health records (EHR)-based triggers (eTriggers) have been used to study diagnostic errors in the emergency department (ED), often with suboptimal performance. Our objective was to investigate incremental value of multi-factor machine learning (ML) approaches to improve eTrigger performance. Methods: Patients presenting to an academic ED were categorized into trigger-positive and trigger-negative using standard trigger (T) definitions: (T1) ED return visits resulting in admission within 10 days; (T2) care escalation from the inpatient unit to the ICU within 24 hours; and (T3) deaths within 24 hours of admission. We trained and evaluated 6 supervised ML models. Results: A total of 124,053 consecutive encounters (5791 T-positive and 118,262 T-negative) were included. Among the T-positive, 4159 (72%) were associated with T1, 1415 (24%) with T2, and 217 (4%) with T3. The T-based positive predictive values (PPV) were 5.2% for T1, 8.2% for T2, and 6.5% for T3. ML models trained and evaluated on balanced training dataset and imbalanced test set had low classification performances (accuracy: 0.72–0.95; PPV: 0.00–0.16; F1-score: 0.00–0.23). Higher performances were observed in balanced test sets (accuracy: 0.80–0.97; PPV: 0.82–1.00; F1-score: 0.79–0.97). Comparing models trained on clinically annotated data with models trained on T-based labels identified other important factors. Conclusions: Utilizing machine learning to refine e-triggers slightly improves the identification of diagnostic errors, as evidenced by an increase in PPV values. We identified new potential factors contributing to ED diagnostic errors. These findings open new avenues to construct or modify more accurate e-triggers for diagnostic error identification.

Importance Diagnostic errors represent a major patient safety concern, with the potential to significantly impact patient outcomes. To address this, various trigger-based strategies have been developed to identify diagnostic errors, aiming to enhance clinical decision-making and improve patient safety. Objective To evaluate the performance of three pre-established triggers (T) in the emergency department (ED) setting and assess their effectiveness in detecting diagnostic errors. Design Consecutive cohort, retrospective observational design. Setting Academic ED with 80 000 annual visits. Participants Adults and children presenting to a single ED in the USA between 1 May 2018 and 1 January 2020. Intervention/outcomes Electronic health records (EHRs) were retrieved and categorised into trigger-positive and trigger-negative cases using the following criteria: T1—unscheduled returnvisits to the ED with admission within 7–10 days of theinitial visit; T2—care escalation from the inpatient unitto the intensive care unit (ICU) within 6, 12 or 24 hoursof ED admission; and T3—all deaths in the ED or within24 hours of ED admission, excluding palliative care. A random sample of trigger-positive cases was reviewed using the SaferDx tool to determine the presence or absence of a diagnostic error. Results A total of 5791 trigger-positive and 118262 trigger-negative cases were identified. Among trigger-positive cases, 4159 (72%) were associated with T1, 1415 (24%) with T2, and 217 (4%) with T3. A preliminary chart review of 462 trigger-positive and 251 trigger-negative cases showed most were error-negative (279 and 217, respectively). Detailed reviews found 32 diagnostic errors among 183 trigger-positive cases, yielding PPVs of 5.4% (T1), 8.9% (T2), and 6.9% (T3). No errors were found in 34 reviewed trigger-negative cases, resulting in a 100% NPV. Sepsis was the most common diagnosis among error-positive cases (n=11, 34.4%). Those with non-specific chief complaints like altered mental status or shortness of breath had higher diagnostic error risk. Conclusion and relevance While previously proposed EHR-based triggers can identify some diagnostic errors, they are insufficient for detecting all cases. To improve error detection performance, we recommend exploring data-driven strategies, such as machine learning techniques, to more effectively identify underlying contributing factors to diagnostic errors and enhance detection accuracy in the ED.

Abstract Objectives There is limited knowledge on how providers and patients in the emergency department (ED) use electronic health records (EHRs) to facilitate the diagnostic process. While EHRs can support diagnostic decision-making, EHR features that are not user-centered may increase the likelihood of diagnostic error. We aimed to identify how EHRs facilitate or impede the diagnostic process in the ED and to identify opportunities to reduce diagnostic errors and improve care quality. Materials and Methods We conducted semistructured interviews with 10 physicians, 15 nurses, and 8 patients across 4 EDs. Data were analyzed using a hybrid thematic analysis approach, which blends deductive (ie, using multiple conceptual frameworks) and inductive coding strategies. A team of 4 coders performed coding. Results We identified 4 themes, 3 at the care team level and 1 at the patient level. At the care team level, the benefits of the EHR in the diagnostic process included (1) customizing features to facilitate diagnostic workup and (2) aiding in communication. However, (3) EHR-driven protocols were found to potentially burden the care process and reliance on asynchronous communication could impede team dynamics. At the patient-level, we found that (4) patient portals facilitated meaningful patient engagement through timely delivery of results. Discussion While EHRs can improve the diagnostic process, they can also impair communication and increase workload. Electronic health record design should leverage provider-created tools to improve usability and enhance diagnostic safety. Conclusions Our findings have important implications for health information technology design and policy. Further work should assess optimal ways to release patient results via the EHR portal.

Introduction Diagnostic processes in the emergency department (ED) involve multiple interactions among individuals who interface with information systems to access and record information. A better understanding of diagnostic processes is needed to mitigate errors. This paper describes a study protocol to map diagnostic processes in the ED as a foundation for developing future error mitigation strategies. Methods and analysis This study of an adult and a paediatric academic ED uses a prospective mixed methods case study design informed by an ED-specific diagnostic decision-making model (the modified ED-National Academies of Sciences, Engineering and Medicine (NASEM) model) and two cognitive theories (dual process theory and distributed cognition). Data sources include audio recordings of patient and care team interactions, electronic health record data, observer field notes and stakeholder interviews. Multiple qualitative analysis methods will be used to explore diagnostic processes in situ, including systems information flow, human–human and human–system interactions and contextual factors influencing cognition. The study has three parts. Part 1 involves prospective field observations of patients with undifferentiated symptoms at high risk for diagnostic error, where each patient is followed throughout the entire care delivery process. Part 2 involves observing individual care team providers over a 4-hour window to capture their diagnostic workflow, team coordination and communication across multiple patients. Part 3 uses interviews with key stakeholders to understand different perspectives on the diagnostic process, as well as perceived strengths and vulnerabilities, in order to enrich the ED-NASEM diagnostic model. Ethics and dissemination The University of Michigan Institutional Review Board approved this study, HUM00156261. This foundational work will help identify strengths and vulnerabilities in diagnostic processes. Further, it will inform the future development and testing of patient, provider and systems-level interventions for mitigating error and improving patient safety in these and other EDs. The work will be disseminated through journal publications and presentations at national and international meetings.