Griffin, Angie

Objectives: Pulmonary function tests (PFTs) are essential for diagnosis, management, and follow-up of a wide range of respiratory conditions as they provide quantitative insights into various aspects of lung function. Precision in ordering and scheduling PFTs, like any test, is crucial as errors can lead to misdiagnoses, delayed treatments, unnecessary retesting, inflated healthcare costs, and potentially compromised patient care. Hence, the aim of this quality improvement initiative was to reduce the PFT ordering error rate to below 5% by redesigning the ordering workflow and implementing a standardized order set tailored to specific clinical scenarios. Methods: The quality improvement initiative was implemented from August 2023 to March 2024. Data was collected from the electronic medical record (EMR) using predefined operational definitions to identify incorrect PFT orders and scheduling errors. A redesigned orderset was created to align with specific clinical presentations, including symptoms, suspected diagnosis, confirmed diagnosis, and presurgical evaluation. Scheduling was also restructured to match the expected testing time for each category. Education sessions were conducted with ordering providers, and ongoing feedback loops were established. Results: Among 575 preintervention PFT orders, 130 (22.6%) contained errors. Following implementation, review of 607 postintervention orders showed a reduction in error rates to 4.1%, representing an ∼80% improvement ( P <0.01). There was no increase in unnecessary testing, supporting the effectiveness of the intervention without unintended negative consequences. Conclusions: Implementation of a knowledge-based CDSS with standardized order sets significantly reduced PFT ordering errors and variability, improving guideline adherence and care quality. Continued refinement and integration with evolving technologies are essential for broader applicability.

Abstract Objective We evaluated the quality and adoption of a large language model (LLM)-based summarization tool for ongoing hospital care. Materials and methods An AI summarization tool was created to provide a “Patient Story”, specialty-specific “Recent Notes”, and “Recent Events” over the previous 24 or 72 hours. We conducted a pragmatic mixed-methods quality assessment at three tertiary-care academic hospitals utilizing (1) Provider Documentation Summarization Quality Instrument (PDSQI-9), (2) utilization analytics, and (3) qualitative end user feedback. The PDSQI-9 included whether summaries were accurate, cited, comprehensible, organized, succinct, non-stigmatizing, synthesized, thorough, and useful. 512 users were given access, from whom 52 participants submitted 205 surveys (10.2% response rate). Results 52 respondents submitted an average of 4.3 surveys (range 1–8). Users rated the tool favorably across all PDSQI-9 domains, with a combined average score of 4.68 (range 4.56–4.78, S.D. 0.67) across the eight domains scored on a 5-point modified Likert scale. Utilization metrics demonstrated strong uptake with frequent views. Positive qualitative feedback revealed cognitive offloading for complex patients and effective summarization of medical problems. Critical feedback showed a need to cross-reference narrative notes to current-state data and lack of detailed specialty-specific summarization. Discussion Generative artificial intelligence has emerged as a potentially transformative technology for generating succinct, verifiable summaries of ongoing care. In this pragmatic implementation, end-users indicated high perceived quality across PDSQI-9 domains. Conclusions An LLM-based summarization tool for ongoing hospitalization care was rated of high quality by diverse clinicians in real-world settings, demonstrated a favorable safety profile, and showed sustained utilization.