Nemati, Shamim

Abstract Objectives Accurate and timely prediction of physiological deterioration in hospitalized patients is essential, but existing deep learning models, which rely solely on structured electronic health record (EHR) data, generate high false-positive rates by missing crucial contextual information in clinical notes. We present DETERIO-LLM, a hybrid prediction model that selectively integrates large language models (LLMs) with a deep learning model (DETERIO) to reclassify borderline-risk alerts using narrative clinical notes. Materials and Methods This retrospective study used a cohort of 1000 inpatients (4.6% deterioration prevalence). DETERIO-LLM selectively analyzed narrative notes for alerts in the uncertainty range (scores 2.5-3.5). Performance was evaluated using sensitivity, positive predictive value (PPV), and F1 score, benchmarked against comparators including the best-established ML model, eCART. Results DETERIO-LLM significantly improved predictive precision over all comparators. At its optimal threshold, the model achieved a PPV of 30.6% and an F1 score of 37.3%, outperforming eCART. This selective integration reduced false-positive alerts by 46.5% within the uncertainty range while maintaining comparable sensitivity. Discussion This study demonstrates a practical, interpretable method for integrating LLM-derived contextual knowledge into existing structured EHR prediction workflows. By focusing LLM analysis on high-uncertainty alerts, DETERIO-LLM substantially improves clinical utility through a marked reduction in false-positive cases, which is critical for mitigating alert fatigue. Conclusion Selective LLM integration with structured EHR models offers a viable, interpretable path to improving early warning systems in clinical care by leveraging narrative context to reduce alert fatigue while enhancing predictive performance.

ABSTRACT Background Systemic infections are a leading cause of hospitalization and death among patients with cirrhosis. Timely and accurate infection identification is essential for both clinical care and the development of predictive models. However, existing methods such as ICD-10 coding are unreliable, and manual chart review is resource-intensive and difficult to scale. This study aimed to develop and validate an automated large language model (LLM)-based approach for infection classification and subtyping in patients with cirrhosis presenting to the emergency department (ED). Method We developed INFEHR (INfection identification and subtyping using Free-text EHR analysis), an LLM-powered pipeline utilizing Claude 3.5 Sonnet to analyze clinical notes from the first 72 hours of admission. Model outputs were compared against a physician-adjudicated gold standard in a cohort of 1,000 encounters from patients with cirrhosis who presented to the ED. Performance was benchmarked against ICD-10 code–based labeling and CDC Adult Sepsis Event criteria. Results INFEHR achieved 94.7% overall accuracy, with 99.5% sensitivity and 92.8% positive predictive value for identifying infection presence, outperforming ICD-10–based classification across all metrics ( p < 0.0001). The model also demonstrated strong performance in classifying pathogen type and infection site. This pipeline processed notes within seconds, offering improvements in efficiency and scalability over manual review. Conclusion INFEHR offers a scalable, reproducible, and accurate method for infection phenotyping in cirrhosis. By overcoming limitations of traditional coding and manual review, it supports high-throughput infection surveillance, improves cohort construction for clinical research, and enables future integration into real-time decision-support tools in hepatology.