Storlie, Curtis B.

Context: Early palliative care can improve end-of-life outcomes, but referrals to palliative care specialists can be delayed in the primary care setting. Objective: This pilot study assessed the effect of a machine-learning algorithm on time to palliative care in a primary care population. Methods: Patients (aged ≥18 years) were eligible if they were empaneled with a primary care provider (PCP) from July 20, 2020, through May 30, 2021. The algorithm evaluated their health records and presented patients who were predicted to have the greatest need for palliative care. Records were then reviewed by palliative care specialists, and patients were randomized in a stepped-wedge fashion to have a referral notification sent to their PCPs if unmet palliative care needs were verified. Time-to-event outcomes were evaluated with Poisson regression models. Results: Of the 127,080 patients evaluated, 934 had their health records presented for review. Some patients were repeatedly presented by the algorithm (total presentations: 1592). In the intervention arm, PCPs were prompted to order a palliative care consultation for 142 patients. The time to 0.1% of the population receiving a palliative care consultation was 60.9 days for the intervention arm versus 71.8 days for the control arm (probability of a shorter time with the intervention, 0.88). Conclusion: A machine-learning algorithm to identify palliative care needs was successfully integrated into a primary care practice. More work is needed to improve the workflow.

1559 Background: Timely access to specialty Palliative Care (SPC) services provides significant benefits to hospitalized patients with cancer, including improvements in quality of life. However, many patients with cancer who could benefit from SPC suffer from lack of timely referral or do not receive SPC services. A previously published trial of an artificial intelligence/machine learning (AI/ML) model to predict need for SPC at Mayo Clinic increased timely SPC consultation of hospitalized patients overall. We sought to review the performance of this algorithm in patients with cancer across an expanded hospitalized population. Methods: The study population consisted of all patients admitted into a Mayo Clinic hospital in Minnesota, Wisconsin, Arizona and Florida between January 2020 and September 2023. Due to the size of the cohort a case-control sample of three controls for every case was created. Patients were considered to be a patient with cancer if they had at least one billing diagnosis up to 1 year prior to their index hospitalization from any of the 5 HCC categories denoting cancer: Metastatic Cancer and Acute Leukemia (HCC 8); Lung and Other Severe Cancers (HCC 9); Lymphoma and Other Cancers (HCC 10); Colorectal, Bladder, and Other Cancers (HCC 11); and Breast, Prostate, and Other Cancers and Tumors (HCC 12). The training data set consisted of 107,076 patient encounters with a total of 8,355,090 time periods of constant risk. An AI/ML model using gradient boosting methods which contained 269 variables (both static and time-varying) of various classes with SPC consultation treated as a time-to-event outcome was trained. Results: Due to the longitudinal nature of the prediction, performance was assessed using the max score AUC (using the max score a patient received during a given encounter prior to an event or discharge) to produce the AUC. The model had an overall AUC of 0.932 (0.929, 0.935 – 95% CI). We saw a decrease in performance of the AI/ML model in the Oncology population with an overall performance of 0.818 (0.806, 0.83). Some of the most influential variables were previous SPC visit, age, metastatic disease, acute leukemia diagnosis, and pain scores. Conclusions: An AI/ML model can effectively predict the need for an inpatient PC consult in a population of hospitalized patients with cancer. [Table: see text]

Abstract We followed 106 349 primary care patients for 22 385 3099 person-days across 21 calendar months and documented 69 breakthrough coronavirus disease 2019 (COVID-19) hospitalizations: 65/102,613 (0.06%) among those fully vaccinated, 3/11 047 (0.03%) among those previously infected, and 1/7,313 (0.01%) among those with both statuses. These data give providers real-world context regarding breakthrough COVID-19 hospitalization risk.

For many classification problems, in addition to providing accurate classification results, very often it is equally important to determine which predictors are contributing to the results, and by how much. This motivates this article, where a new sensitivity analysis framework is developed for classification problems to address this issue. The effectiveness of this framework is illustrated through simulation studies and application to real data. Notice that this framework can be coupled with different classification methods. However, in practice it is recommended to pair it with a classification method called Bayesian smoothing spline ANOVA probit regression (BSSANOVA). When compared with other existing methods through numerical experiments, BSSANOVA performs extremely well for both classification and sensitivity analysis.

Abstract Background: Palliative care is a medical specialty centered on improving the quality of life (QOL) of patients with complex or life-threatening illnesses. The need for Palliative care is increasing and with that the rigorous testing of triage tools that can be used quickly and reliably to identify patients that may benefit from palliative care. Methods: To that Aim we will conduct a two-armed stepped-wedge cluster randomized trial rolled out to two inpatient hospitals to evaluate whether a machine learning algorithm accurately identifies patients who may benefit from a comprehensive review by a palliative care specialist, and decreases time to receiving a palliative care consult in hospital. This is a single-center study which will be conducted from August 2019 to November 2020 at Saint Mary’s Hospital & Methodist Hospital, Mayo Clinic, Rochester, Minnesota. Clusters will be nursing units which will be chosen to be a mix of complex patients from Cardiology, Critical Care and Oncology and had previously established relationships with palliative medicine. The stepped wedge design will have 12 units allocated to a design matrix of 5 treatment wedges. Each wedge will last 75 days resulting in a study period of 12 months unless otherwise specified. Data will be analyzed with Bayesian hierarchical models with credible intervals denoting statistical significance.Discussion: This intervention offers a pragmatic approach to delivering specialty palliative care to hospital patients in need using machine learning, thereby leading to high value care and improved outcomes. It is not enough for AI to be utilized by simply publishing research showing predictive performance; Clinical trials demonstrating better outcomes are critically needed. Furthermore, the deployment of an AI algorithm is a complex process that requires multiple teams with varying skill sets. To evaluate a deployed AI, a pragmatic clinical trial can accommodate the difficulties of clinical practice whilst retaining scientific rigor.Trial registration NCT03976297 (ClinicalTrials.gov), registered on June 6, 2019 prior to trial start.

Abstract Objective Access to palliative care (PC) is important for many patients with uncontrolled symptom burden from serious or complex illness. However, many patients who could benefit from PC do not receive it early enough or at all. We sought to address this problem by building a predictive model into a comprehensive clinical framework with the aims to (i) identify in-hospital patients likely to benefit from a PC consult, and (ii) intervene on such patients by contacting their care team. Materials and Methods Electronic health record data for 68 349 inpatient encounters in 2017 at a large hospital were used to train a model to predict the need for PC consult. This model was published as a web service, connected to institutional data pipelines, and consumed by a downstream display application monitored by the PC team. For those patients that the PC team deems appropriate, a team member then contacts the patient’s corresponding care team. Results Training performance AUC based on a 20% holdout validation set was 0.90. The most influential variables were previous palliative care, hospital unit, Albumin, Troponin, and metastatic cancer. The model has been successfully integrated into the clinical workflow making real-time predictions on hundreds of patients per day. The model had an “in-production” AUC of 0.91. A clinical trial is currently underway to assess the effect on clinical outcomes. Conclusions A machine learning model can effectively predict the need for an inpatient PC consult and has been successfully integrated into practice to refer new patients to PC.