Tao, Cui

ABSTRACT Objective To comprehensively evaluate the validity of ICD-10-CM codes for both prevalent diagnoses and less common diseases, and to assess the performance of a large language model (LLM)-based system in validating these codes. Materials and Methods This retrospective study analyzed hospital admissions from the Medical Information Mart for Intensive Care (MIMIC-IV) database. We developed a validated LLM-based system using GPT-4o, refined through iterative prompt engineering, to assess ICD-10-CM code validity. We measured the PPV of ICD-10-CM codes, PPV of principal and secondary diagnoses, and the performance of an LLM-based system in code validation. Results Among 865,079 assigned codes, the PPV was 84.6% (95% CI, 84.5%-84.6%). Principal diagnoses had a PPV of 93.9% (95% CI, 93.7%-94.1%), while secondary diagnoses had a PPV of 83.8% (95% CI, 83.7%-83.9%). The LLM system demonstrated high performance in validating ICD codes, achieving 93.6% accuracy, 95.4% sensitivity and 85.2% specificity. Among correctly assigned secondary diagnoses, the majority (67.9%) represented historical or baseline conditions, while 32.1% reflected active conditions that deviated from baseline status; 22.3% of these emerged after hospital admission. PPV decreases with later diagnosis positions, with the largest decline occurring between principal and secondary diagnoses. Discussion and Conclusion In this large-scale evaluation, ICD-10-CM codes exhibited generally high accuracy, though variability existed by position and condition type. A validated LLM system performed comparably to physician review and offers a scalable means to improve coding accuracy. These findings support the potential for integrating LLM-based auditing into routine workflows to strengthen the quality of administrative and research data.

Abstract Background Dynamically predicting patients at risk of Alzheimer's disease (AD) is crucial for timely treatment and care in precision medicine. Method We present a novel architecture, a Kolmogorov‐Arnold networks (KANs)‐based joint prediction model of longitudinal and survival data (JM‐KAN). Real‐life data of patients with Mild Cognitive Impairment (MCI) with no prior AD diagnosis from National Alzheimer's Coordinating Center (NACC) dataset was utilized for individualized predictions. We built joint models with landmark times every 1 year up to 4 years after the MCI diagnosis, and predicted survival outcomes one year after each landmark (i.e. 1, 2, 3, and 4 years). Integrated AUC (iAUC), and integrated Brier score (iBS) were measured for existing prediction models and our novel architecture. Result Out of 2711 unique patients with MCI, 821 patients progressed to develop AD. Mean time‐to‐event in years was 3.19 (standard deviation (SD): 2.29). The number of visits prior to AD diagnosis was 4.03 (SD: 2.41). With iAUC of 0.789, iBS of 0.118, JM‐KAN has both the highest iAUC and lowest iBS among joint models, which indicates superiority in discrimination and overall performance. For individualized predictions, the model successfully predicted both longitudinal risk factors and survival probability, as new information came in every year. Conclusion Using neural network‐based Artificial Intelligence (AI) model, we demonstrated that this architecture improves upon prior methods in dynamically updating predictions of AD as new longitudinal data becomes available.