Minteer, Sarah A.

ABSTRACT Introduction Sustaining interventions implemented as part of pragmatic clinical trials requires proactive planning, but how to approach this process remains understudied. The Non‐pharmacological Options in Hospital‐based and Rehabilitation pain Management stepped‐wedge cluster‐randomized pragmatic trial tested an electronic health record‐based educational bundle (known as the Healing After Surgery [HAS] initiative) that encouraged the incorporation of non‐pharmacological pain care in perioperative pain management. The HAS initiative was implemented in multiple surgical practices and hospital sites. One year prior to trial completion, a sustainment committee was established to support the transition of intervention components to clinical ownership and to support posttrial sustainment of the HAS initiative. Methods We reviewed meeting minutes from biweekly sustainment committee meetings and conducted debriefing sessions with committee members (informed by the Clinical Sustainability Assessment Tool) to identify sustainment strategies and challenges, which were then organized into meaningful themes. Results The following six themes emerged from meeting minutes and debriefings: (1) automation of low‐touch components, (2) reliance on internal champions, (3) intentional handoff communication, (4) institutional attention, (5) relaxed fidelity, and (6) continuous evaluation. Together, they highlight the importance of early, structured planning and adaptable implementation strategies. Conclusions Sustainment of the HAS initiative required extensive communication, adaptation, and stakeholder engagement across diverse institutional contexts. Proactive sustainment planning during trial design may help ensure interventions are successfully integrated into routine clinical practice posttrial.

AbstractElectronic patient-reported outcome measures (ePROMs) can improve care for people with cancer, but effectiveness hinges on well-supported integration in clinical settings.We evaluated clinician use of specific clinical decision support (CDS) tools in the electronic health record (EHR) designed to facilitate timely, clinically appropriate responses to ePROM scores for six symptoms commonly experienced by cancer patients.The parent pragmatic trial, which took place at Mayo Clinic (Rochester, Minnesota, United States) and its affiliated community health care system between March 2019 and January 2023, evaluated the population-level effectiveness and implementation of an ePROM surveillance and EHR-facilitated collaborative care symptom management intervention. The present evaluation used a case study approach with four data sources: (1) clinician interactions with CDS tools abstracted from the EHR; (2) clinician notes identified with an institution-specific textual search tool; (3) qualitative interviews and group discussions with care teams; and (4) administrative records reviewed to identify training and outreach to care teams.EHR metrics showed very low adoption of CDS tools including alerts and symptom-specific order sets, despite educational outreach and information technology support provided to clinical care teams. Qualitative findings revealed that CDS use was not easy to integrate into busy clinical workflows and highlighted clinician perceptions that the collaborative care intervention provided additional patient support that reduced clinicians' need to utilize CDS tools. They also highlight the importance of contextual factors, including institutional priorities and EHR changes.This pragmatic clinical trial case study found limited adoption of EHR CDS tools that had been developed to increase clinicians' awareness of and responses to ePROM data. Findings suggest the need to align clinician and organizational implementation strategies, simplify CDS tools to fit practice expectations, and identify and address contextual factors that could undercut strategies like education and peer support. This may be especially important for teams who aim to iteratively evaluate and refine CDS and implementation strategies for multicomponent interventions or introduce new strategies that are responsive to barriers while maintaining scalability.

Abstract Objectives This study aimed to evaluate critical care provider perspectives about diagnostic practices for rare and atypical infections and the potential for using artificial intelligence (AI) as a decision support system (DSS). Methods We conducted an anonymous web-based survey among critical care providers at Mayo Clinic Rochester between November 25, 2023, and January 15, 2024, to evaluate their experience with rare and atypical infection diagnostic processes and AI-based DSSs. We also assessed the perceived usefulness of AI-based DSSs, their potential impact on improving diagnostic practices for rare and atypical infections, and the perceived risks and benefits of their use. Results A total of 47/143 providers completed the survey. Thirty-eight out of 47 agreed that there was a delay in diagnosing rare and atypical infections. Among those who agreed, limited assessment of specific patient factors and failure to consider them were the most frequently cited important contributing factors (33/38). Thirty-eight out of 47 reported familiarity with the AI-based DSS applications available to critical care providers. Less than half (18/38) thought AI-based DSSs often provided valuable insights into patient care, but almost three-quarters (34/47) thought AI-based DDSs often provided valuable insight when specifically asked about their ability to improve the diagnosis of rare and atypical infections. All respondents rated reliability as important in enhancing the perceived utility of AI-based DSSs (47/47) and almost all rated interpretability and integration into the workflow as important (45/47). The primary concern about implementing an AI-based DSS in this context was alert fatigue (44/47). Conclusion Most critical care providers perceive that there are delays in diagnosing rare infections, indicating inadequate assessment and consideration of the diagnosis as the major contributors. Reliability, interpretability, workflow integration, and alert fatigue emerged as key factors impacting the usability of AI-based DSS. These findings will inform the development and implementation of an AI-based diagnostic algorithm to aid in identifying rare and atypical infections.

AbstractObjectivesTo report lessons from integrating the methods and perspectives of clinical informatics (CI) and implementation science (IS) in the context of Improving the Management of symPtoms during and following Cancer Treatment (IMPACT) Consortium pragmatic trials.Materials and MethodsIMPACT informaticists, trialists, and implementation scientists met to identify challenges and solutions by examining robust case examples from 3 Research Centers that are deploying systematic symptom assessment and management interventions via electronic health records (EHRs). Investigators discussed data collection and CI challenges, implementation strategies, and lessons learned.ResultsCI implementation strategies and EHRs systems were utilized to collect and act upon symptoms and impairments in functioning via electronic patient-reported outcomes (ePRO) captured in ambulatory oncology settings. Limited EHR functionality and data collection capabilities constrained the ability to address IS questions. Collecting ePRO data required significant planning and organizational champions adept at navigating ambiguity.DiscussionBringing together CI and IS perspectives offers critical opportunities for monitoring and managing cancer symptoms via ePROs. Discussions between CI and IS researchers identified and addressed gaps between applied informatics implementation and theory-based IS trial and evaluation methods. The use of common terminology may foster shared mental models between CI and IS communities to enhance EHR design to more effectively facilitate ePRO implementation and clinical responses.ConclusionImplementation of ePROs in ambulatory oncology clinics benefits from common understanding of the concepts, lexicon, and incentives between CI implementers and IS researchers to facilitate and measure the results of implementation efforts.

PURPOSE The enhanced, electronic health record (EHR)–facilitated cancer symptom control (E2C2) trial is a cohort cluster-randomized, stepped-wedge, hybrid type II trial that leverages EHR systems to facilitate a collaborative care model (CCM) approach with the goal of improving cancer symptom management. Understanding factors that influence care team adoption of EHR systems remains a critical understudied area of research. This study examines how oncology care teams' perceptions regarding the feasibility, acceptability, and appropriateness of E2C2 EHR systems preimplementation were associated with adoption 3 months after implementation and characterizes differences in adoption by individual- and system-level factors. METHODS Care team members completed an electronic survey before and 3 months after implementation of E2C2 for their respective sequence. Adoption was defined as frequency of use to statements aligned with care team–directed EHR systems designed to facilitate CCM approaches. Chi-square tests assessed differences in adoption while logistic regression models estimated associations between baseline mean scores of acceptability, feasibility, and appropriateness on care team adoption at 3 months. RESULTS Results from 94 care team members (37.2% oncologists, 72.6% female, 55.3% in their role for 6+ years) found that adoption rates ranged from 48.9% to 71.7%, with significant differences observed by location (community-based health care systems v tertiary medical center) and professional role. Adjusting for professional role, care team members reporting higher levels of perceived acceptability and appropriateness at baseline had greater odds of adopting EHR systems at 3 months. CONCLUSION EHR systems perceived as acceptable and appropriate are more likely to be adopted by oncology care teams in our sample. Future implementation efforts should consider tailored strategies to facilitate adoption of EHR systems designed to promote CCM-based approaches to improve cancer symptom management.

Abstract Background The electronic health record (EHR) presents new opportunities for the timely identification of patients at high risk of critical illness and the implementation of preventive strategies. This study aims to externally validate an EHR‐based Elders Risk Assessment (ERA) score to identify older patients at high risk of future critical illness during a primary care visit. Methods This historical cohort study included patients aged ≥65 years who had primary care visits at Mayo Clinic Rochester, MN, between July 2019 and December 2021. The ERA score at the time of the primary care visit was used to predict critical illness, defined as death or ICU admission within 1 year of the visit. Results A total of 12,885 patients were included in the analysis. The median age at the time of the primary care visit was 75 years, with 44.6% being male. 93.7% of participants were White, and 64.2% were married. The median (25th, 75th percentile) ERA score was 4 (0, 9). 11.3% of study participants were admitted to the ICU or died within 1 year of the visit. The ERA score predicted critical illness within 1 year of a primary care visit with an area under the receiver operating characteristic curve of 0.84 (95% CI 0.83–0.85), which indicates good discrimination. An ERA score of 9 was identified as optimal for implementing and testing potential preventive strategies, with the odds ratio of having the primary outcome in patients with ERA score ≥9 being 11.33 (95%CI 9.98–12.87). Conclusions This simple EHR‐based risk assessment model can predict critical illness within 1 year of primary care visits in older patients. The findings of this study can serve as a basis for testing and implementation of preventive strategies to promote the well‐being of older adults at risk of critical illness and its consequences.

Objective: Given limited critical care resources and an aging population, early interventions to prevent critical illness are vital. In this work, we measured post-implementation outcomes after introducing a novel electronic scoring system (Elders Risk Assessment—ERA) and a risk-factor checklist, Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), to detect older patients at high risk of critical illness in a primary care setting. Methods: The study was conducted at a family medicine clinic in Kasson, MN. The ADAPT-ITT framework was used to modify the CERTAIN checklist for primary care during 2 co-design workshops involving interdisciplinary clinicians, held in April 2023. The ERA score and modified CERTAIN checklist were implemented between May and July 2023 and identify and assess all patients age ≥60 years at risk of critical illness during their primary care visits. Implementation outcomes were evaluated at the end of the study via an anonymous survey and EHR data extraction. Results: Fourteen clinicians participated in 2 co-design workshops. A total of 19 clinicians participated in a post-pilot survey. All survey items were rated on a 5-point Likert type scale. Mean acceptability of the ERA score and checklist was rated 3.35 (SD = 0.75) and 3.09 (SD = 0.64), respectively. Appropriateness had a mean rating of 3.38 (SD = 0.82) for the ERA score and 3.19 (SD = 0.59) for the checklist. Mean feasibility was rated 3.38(SD = 0.85) and 2.92 (SD = 0.76) for the ERA score and checklist, respectively. The adoption rate was 50% (19/38) among clinicians, but the reach was low at 17% (49/289) of eligible patients. Conclusions: This pilot study evaluated the implementation of an intervention that introduced the ERA score and CERTAIN checklist into a primary care practice. Results indicate moderate acceptability, appropriateness, and feasibility of the ERA score, and similar ratings for the checklist, with slightly lower feasibility. While checklist adoption was moderate, reach was limited, indicating inconsistent use. Recommendations: We plan to use the open-ended resurvey responses to further modify the CERTAIN-FM checklist and implementation process. The ADAPT-ITT framework is a useful model for adapting the checklist to meet the primary care clinician needs.

Diagnostic error or delay (DEOD) is common in the acute care setting and results in poor patient outcomes. Many factors contribute to DEOD, but little is known about how contributors may differ across acute care areas and professional roles. As part of a sequential exploratory mixed methods research study, we surveyed acute care clinical stakeholders about the frequency with which different factors contribute to DEOD. Survey respondents could also propose solutions in open text fields. N = 220 clinical stakeholders completed the survey. Care Team Interactions, Systems and Process, Patient, Provider, and Cognitive factors were perceived to contribute to DEOD with similar frequency. Organization and Infrastructure factors were perceived to contribute to DEOD significantly less often. Responses did not vary across acute care setting. Physicians perceived Cognitive factors to contribute to DEOD more frequently compared to those in other roles. Commonly proposed solutions included: technological solutions, organization level fixes, ensuring staff know and are encouraged to work to the full scope of their role, and cultivating a culture of collaboration and respect. Multiple factors contribute to DEOD with similar frequency across acute care areas, suggesting the need for a multi-pronged approach that can be applied across acute care areas.

AbstractIntroductionMany evidence‐based programs (EBPs) have been determined in randomized controlled trials to be effective, but few studies explore the real‐world effectiveness of EBPs implemented in the natural community setting. Our study evaluated whether a novel linked infrastructure would enable such insights and continuous improvement as part of a learning healthcare‐community bridged “wellcare” ecosystem.MethodsWe created a secure, web‐based data entry and storage platform with a network of Minnesota community‐based organizations to record EBP participants' demographics and attendance, and program details. We then linked participant's information to their Rochester Epidemiology Project (REP) medical records. With this infrastructure, we conducted a proof of concept, retrospective cohort study by matching EBP participants to REP controls and comparing medical record‐documented outcomes over 1 year follow‐up.ResultsWe successfully linked EBP participant records with medical records in 77.6% of cases, and the infrastructure proved feasible and scalable. Still, key challenges remain in obtaining participant consent for data sharing. Upfront resource investments and the availability of REP‐like warehouses limit generalizability. Optimal learning will be improved by enhancements that better track program fidelity. Our pilot study established a proof‐of‐concept, but sample sizes (n = 99 for falls prevention and n = 97 chronic disease/pain management EBP completers) were too small to detect significant differences in hospital admittance as compared to matched controls for either EBP group, (OR = 0.66[0.36, 1.19]) and (OR = 0.81[0.43, 1.54]), respectively. Events were too rare to gather meaningful information about effects on fall rates.ConclusionsOur pilot demonstrates the feasibility of developing an online infrastructure that connects information from community leaders with medical record documented health outcomes, bridging the knowledge gap between community programs and the health care system. Insights gleaned from our infrastructure can be used to continuously shape community program delivery to reduce the need for formal health care services.