Huschka, Todd R.

Objectives: Electronic health records (EHR)-based triggers (eTriggers) have been used to study diagnostic errors in the emergency department (ED), often with suboptimal performance. Our objective was to investigate incremental value of multi-factor machine learning (ML) approaches to improve eTrigger performance. Methods: Patients presenting to an academic ED were categorized into trigger-positive and trigger-negative using standard trigger (T) definitions: (T1) ED return visits resulting in admission within 10 days; (T2) care escalation from the inpatient unit to the ICU within 24 hours; and (T3) deaths within 24 hours of admission. We trained and evaluated 6 supervised ML models. Results: A total of 124,053 consecutive encounters (5791 T-positive and 118,262 T-negative) were included. Among the T-positive, 4159 (72%) were associated with T1, 1415 (24%) with T2, and 217 (4%) with T3. The T-based positive predictive values (PPV) were 5.2% for T1, 8.2% for T2, and 6.5% for T3. ML models trained and evaluated on balanced training dataset and imbalanced test set had low classification performances (accuracy: 0.72–0.95; PPV: 0.00–0.16; F1-score: 0.00–0.23). Higher performances were observed in balanced test sets (accuracy: 0.80–0.97; PPV: 0.82–1.00; F1-score: 0.79–0.97). Comparing models trained on clinically annotated data with models trained on T-based labels identified other important factors. Conclusions: Utilizing machine learning to refine e-triggers slightly improves the identification of diagnostic errors, as evidenced by an increase in PPV values. We identified new potential factors contributing to ED diagnostic errors. These findings open new avenues to construct or modify more accurate e-triggers for diagnostic error identification.

Importance Diagnostic errors represent a major patient safety concern, with the potential to significantly impact patient outcomes. To address this, various trigger-based strategies have been developed to identify diagnostic errors, aiming to enhance clinical decision-making and improve patient safety. Objective To evaluate the performance of three pre-established triggers (T) in the emergency department (ED) setting and assess their effectiveness in detecting diagnostic errors. Design Consecutive cohort, retrospective observational design. Setting Academic ED with 80 000 annual visits. Participants Adults and children presenting to a single ED in the USA between 1 May 2018 and 1 January 2020. Intervention/outcomes Electronic health records (EHRs) were retrieved and categorised into trigger-positive and trigger-negative cases using the following criteria: T1—unscheduled returnvisits to the ED with admission within 7–10 days of theinitial visit; T2—care escalation from the inpatient unitto the intensive care unit (ICU) within 6, 12 or 24 hoursof ED admission; and T3—all deaths in the ED or within24 hours of ED admission, excluding palliative care. A random sample of trigger-positive cases was reviewed using the SaferDx tool to determine the presence or absence of a diagnostic error. Results A total of 5791 trigger-positive and 118262 trigger-negative cases were identified. Among trigger-positive cases, 4159 (72%) were associated with T1, 1415 (24%) with T2, and 217 (4%) with T3. A preliminary chart review of 462 trigger-positive and 251 trigger-negative cases showed most were error-negative (279 and 217, respectively). Detailed reviews found 32 diagnostic errors among 183 trigger-positive cases, yielding PPVs of 5.4% (T1), 8.9% (T2), and 6.9% (T3). No errors were found in 34 reviewed trigger-negative cases, resulting in a 100% NPV. Sepsis was the most common diagnosis among error-positive cases (n=11, 34.4%). Those with non-specific chief complaints like altered mental status or shortness of breath had higher diagnostic error risk. Conclusion and relevance While previously proposed EHR-based triggers can identify some diagnostic errors, they are insufficient for detecting all cases. To improve error detection performance, we recommend exploring data-driven strategies, such as machine learning techniques, to more effectively identify underlying contributing factors to diagnostic errors and enhance detection accuracy in the ED.

Objective In-person healthcare delivery is rapidly changing with a shifting employment landscape and technological advances. Opportunities to care for patients in more efficient ways include leveraging technology and focusing on caring for patients in the right place at the right time. We aim to use computer modelling to understand the impact of interventions, such as virtual consultation, on hospital census for referring and referral centres if non-procedural patients are cared for locally rather than transferred. Patients and methods We created computer modelling based on 25 138 hospital transfers between June 2019 and June 2022 with patients originating at one of 17 community-based hospitals and a regional or academic referral centre receiving them. We identified patients that likely could have been cared for at a community facility, with attention to hospital internal medicine and cardiology patients. The model was run for 33 500 days. Results Approximately 121 beds/day were occupied by transferred patients at the academic centre, and on average, approximately 17 beds/day were used for hospital internal medicine and nine beds/day for non-procedural cardiology patients. Typical census for all internal medicine beds is approximately 175 and for cardiology is approximately 70. Conclusion Deferring transfers for patients in favour of local hospitalisation would increase the availability of beds for complex care at the referral centre. Potential downstream effects also include increased patient satisfaction due to proximity to home and viability of the local hospital system/economy, and decreased resource utilisation for transfer systems.

Abstract Background: Individuals from diverse racial/ethnic and disadvantaged backgrounds are underrepresented in cancer clinical trials, compounding the difficulty of adequately understanding, addressing, and reversing health inequities. Social determinants of health (SDOH) factors likely contribute to disparities in cancer trial enrollment. While the presence of SDOH has shown to be associated with clinical trial participation, few studies have explored the role of SDOH in shaping beliefs around the benefits of clinical trials. Thus, this study aims to identify SDOH factors associated with beliefs around the benefits of clinical trial participation. Methods: This is a cross-sectional study of data from the Community Health Assets and Resilience Measures survey. The survey was administered in Fall 2021 to community members (N=3593; 98% > 50 years of age, 68% male, 52% racial/ethnic minority) residing in three states within the Mayo Clinic catchment area. Survey items assessed SDOH factors including financial instability, childcare, transportation, food insecurity, safety, literacy, social isolation, and perceived discrimination. Participant beliefs around the benefits of clinical trial participation was measured by rating level of agreement to 4 statements: ‘People in my community can benefit from participating in clinical trials’, ‘Participation in clinical trials benefits society’, ‘Participation in clinical trials is risky’, ‘I would personally benefit’, and ‘People in my community would benefit’. Response options were dichotomized as ‘agree’ and ‘disagree’ and adjusted stepwise logistic regression analyses were performed to assess factors associated with beliefs about clinical trials. Results: Race/ethnicity was significantly associated with all beliefs with Indigenous populations being significantly more likely to agree that people in their community could benefit from clinical trials, clinical trials benefit society, participation in clinical trials is risky, and that they would personally benefit. In adjusted analyses, financial instability, food insecurity, transportation, childcare, safety, social isolation, literacy, and discrimination were significantly associated with beliefs about the benefits of clinical trial participation. Neighborhood safety was most strongly associated with the belief that ‘People in my community can benefit from participating in clinical trials’ and ‘I would personally benefit from clinical trials’. Health literacy was most strongly associated with the belief that clinical trials benefit society and the community. Finally, the experience of 'people act as if they think you are not smart' (discrimination) was most strongly associated with the belief that clinical trials are risky. Conclusion: Clinical trials are the cornerstone to advancing equitable cancer care. Innovative, multifaceted, community-driven strategies are needed that address negative beliefs around the benefits of clinical trial participation to address existing recruitment and retention shortcomings. Citation Format: Jessica D. Austin, Todd Huschka, Amelia Barwise, Megan Allyse, Sean Phelan. The role of social determinants of health on beliefs around the benefits of clinical trials [abstract]. In: Proceedings of the 16th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2023 Sep 29-Oct 2;Orlando, FL. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2023;32(12 Suppl):Abstract nr A077.

AbstractBackgroundHospitals face the challenge of managing demand for limited computed tomography (CT) resources from multiple patient types while ensuring timely access.MethodsA discrete event simulation model was created to evaluate CT access time for emergency department (ED) patients at a large academic medical center with six unique CT machines that serve unscheduled emergency, semi‐scheduled inpatient, and scheduled outpatient demand. Three operational interventions were tested: adding additional patient transporters, using an alternative creatinine lab, and adding a registered nurse dedicated to monitoring CT patients in the ED.ResultsAll interventions improved access times. Adding one or two transporters improved ED access times by up to 9.8 minutes (Mann‐Whitney (MW) CI: [−11.0,−8.7]) and 10.3 minutes (MW CI [−11.5, −9.2]). The alternative creatinine and RN interventions provided 3‐minute (MW CI: [−4.0, −2.0]) and 8.5‐minute (MW CI: [−9.7, −8.3]) improvements.ConclusionsAdding one transporter provided the greatest combination of reduced delay and ability to implement. The projected simulation improvements have been realized in practice.

The prevailing first‐come‐first‐served approach to outpatient appointment scheduling ignores differing urgency levels, leading to unnecessarily long waits for urgent patients. In data from a partner healthcare organization, we found in some departments that urgent patients were inadvertently waiting longer for an appointment than non‐urgent patients. This study develops a capacity allocation optimization methodology that reserves appointment slots based on urgency in a complicated, integrated care environment where multiple specialties serve multiple types of patients. This optimization reallocates network capacity to limit access delays (indirect waiting times) for initial and downstream appointments differentiated by urgency. We formulate this problem as a queueing network optimization and approximate it via deterministic linear optimization to simultaneously smooth workloads and guarantee access delay targets. In a case study of our industry partner we demonstrate the ability to (i) reduce urgent patient mean access delay by 27% with only a 7% increase in mean access delay for non‐urgent patients, and (ii) increase throughput by 31% with the same service levels and overtime.

Discrete-event simulation (DES) is an effective tool to for analyzing and improving healthcare processes. In this chapter we discuss the use of simulation to improve patient flow at an outpatient procedure center (OPC) at Mayo Clinic. The OPC addressed is the Pain Clinic, which was faced with high patient volumes in a new, untested facility. Simulation was particularly useful due to the uncertain patient procedure and recovery times. We discuss the simulation process and show how it helped reduce patient waiting time while ensuring the clinic could meet its target patient volumes.

Uncertainty in the duration of surgical procedures can cause long patient wait times, poor utilization of resources, and high overtime costs. We compare several heuristics for scheduling an Outpatient Procedure Center. First, a discrete event simulation model is used to evaluate how 12 different sequencing and patient appointment time‐setting heuristics perform with respect to the competing criteria of expected patient waiting time and expected surgical suite overtime for a single day compared with current practice. Second, a bi‐criteria genetic algorithm (GA) is used to determine if better solutions can be obtained for this single day scheduling problem. Third, we investigate the efficacy of the bi‐criteria GA when surgeries are allowed to be moved to other days. We present numerical experiments based on real data from a large health care provider. Our analysis provides insight into the best scheduling heuristics, and the trade‐off between patient and health care provider‐based criteria. Finally, we summarize several important managerial insights based on our findings.

Operating room (OR) scheduling is an important operational problem for most hospitals. In this study, we present a novel two-stage stochastic mixed-integer programming model to minimize total expected operating cost given that scheduling decisions are made before the resolution of uncertainty in surgery durations. We use this model to quantify the benefit of pooling ORs as a shared resource and to illustrate the impact of parallel surgery processing on surgery schedules. Decisions in our model include the number of ORs to open each day, the allocation of surgeries to ORs, the sequence of surgeries within each OR, and the start time for each surgeon. Realistic-sized instances of our model are difficult or impossible to solve with standard stochastic programming techniques. Therefore, we exploit several structural properties of the model to achieve computational advantages. Furthermore, we describe a novel set of widely applicable valid inequalities that make it possible to solve practical instances. Based on our results for different resource usage schemes, we conclude that the impact of parallel surgery processing and the benefit of OR pooling are significant. The latter may lead to total cost reductions between 21% and 59% on average.

The allocation of surgeries to operating rooms (ORs) is a challenging combinatorial optimization problem. There is also significant uncertainty in the duration of surgical procedures, which further complicates assignment decisions. In this paper, we present stochastic optimization models for the assignment of surgeries to ORs on a given day of surgery. The objective includes a fixed cost of opening ORs and a variable cost of overtime relative to a fixed length-of-day. We describe two types of models. The first is a two-stage stochastic linear program with binary decisions in the first stage and simple recourse in the second stage. The second is its robust counterpart, in which the objective is to minimize the maximum cost associated with an uncertainty set for surgery durations. We describe the mathematical models, bounds on the optimal solution, and solution methodologies, including an easy-to-implement heuristic. Numerical experiments based on real data from a large health-care provider are used to contrast the results for the two models and illustrate the potential for impact in practice. Based on our numerical experimentation, we find that a fast and easy-to-implement heuristic works fairly well, on average, across many instances. We also find that the robust method performs approximately as well as the heuristic, is much faster than solving the stochastic recourse model, and has the benefit of limiting the worst-case outcome of the recourse problem.