Citations

Abstract Objectives To comprehensively evaluate the validity of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes for both prevalent diagnoses and less common diseases, and to assess the performance of a large language model (LLM)-based system in validating these codes. Materials and Methods This retrospective study analyzed hospital admissions from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. We developed a validated LLM-based system using GPT-4o, refined through iterative prompt engineering, to assess ICD-10-CM code validity. We measured the positive predictive value (PPV) of ICD-10-CM codes, PPV of principal and secondary diagnoses, and the performance of an LLM-based system in code validation. Results Among 865 079 assigned codes, the PPV was 84.6% (95% CI, 84.5%-84.6%). Principal diagnoses had a PPV of 93.9% (95% CI, 93.7%-94.1%), while secondary diagnoses had a PPV of 83.8% (95% CI, 83.7%-83.9%). The LLM system demonstrated high performance in validating ICD codes, achieving 93.6% accuracy, 95.4% sensitivity, and 85.2% specificity. Among correctly assigned secondary diagnoses, the majority (67.9%) represented historical or baseline conditions, while 32.1% reflected active conditions that deviated from baseline status; 22.3% of these emerged after hospital admission. PPV decreases with later diagnosis positions, with the largest decline occurring between principal and secondary diagnoses. Discussion and Conclusion In this large-scale evaluation, ICD-10-CM codes exhibited generally high accuracy, though variability existed by position and condition type. A validated LLM system performed comparably to physician review and offers a scalable means to improve coding accuracy. These findings support the potential for integrating LLM-based auditing into routine workflows to strengthen the quality of administrative and research data.

ABSTRACT Objective To comprehensively evaluate the validity of ICD-10-CM codes for both prevalent diagnoses and less common diseases, and to assess the performance of a large language model (LLM)-based system in validating these codes. Materials and Methods This retrospective study analyzed hospital admissions from the Medical Information Mart for Intensive Care (MIMIC-IV) database. We developed a validated LLM-based system using GPT-4o, refined through iterative prompt engineering, to assess ICD-10-CM code validity. We measured the PPV of ICD-10-CM codes, PPV of principal and secondary diagnoses, and the performance of an LLM-based system in code validation. Results Among 865,079 assigned codes, the PPV was 84.6% (95% CI, 84.5%-84.6%). Principal diagnoses had a PPV of 93.9% (95% CI, 93.7%-94.1%), while secondary diagnoses had a PPV of 83.8% (95% CI, 83.7%-83.9%). The LLM system demonstrated high performance in validating ICD codes, achieving 93.6% accuracy, 95.4% sensitivity and 85.2% specificity. Among correctly assigned secondary diagnoses, the majority (67.9%) represented historical or baseline conditions, while 32.1% reflected active conditions that deviated from baseline status; 22.3% of these emerged after hospital admission. PPV decreases with later diagnosis positions, with the largest decline occurring between principal and secondary diagnoses. Discussion and Conclusion In this large-scale evaluation, ICD-10-CM codes exhibited generally high accuracy, though variability existed by position and condition type. A validated LLM system performed comparably to physician review and offers a scalable means to improve coding accuracy. These findings support the potential for integrating LLM-based auditing into routine workflows to strengthen the quality of administrative and research data.

Abstract Glioblastoma is highly vascularized with increased perfusion and prognosis remains dismal, despite improved diagnosis and aggressive multimodal treatment. The current standard of care is maximal surgical resection when feasible, followed by adjuvant radiation with concurrent chemotherapy, temozolomide. However, treatment resistance and tumor recurrence pose a considerable challenge for stratifying patients for optimal treatment planning. Current radiological assessment of glioblastoma therapy response predominantly relies on morphological measures; that may not accurately capture early microscopic changes or therapeutic efficacy. We have developed and validated an advanced imaging method, Arterial Spin Labeling (ASL) MRI that enable noninvasive spatial assessment of the entire tumor. ASL-MRI uses water within the blood as an endogenous tracer and measures perfusion without the need for exogenous gadolinium-based contrast agent. The objective of this study was to investigate ASL measured quantitative perfusion after surgery to predict survival in glioblastoma patients undergoing the standard of care chemoradiation treatment. In this prospective, IRB-approved study, twenty-three adults with glioblastoma participated for longitudinal MRI. Progression-free survival (PFS) and overall survival (OS) were recorded for each patient. A baseline MRI scan was performed prior to chemoradiation treatment and after surgery utilizing a 3D ASL sequence. All structural images were co-registered to T1 post-contrast images. Tumor regions of interest (ROIs) such as enhancing tumor (ET), tumor core, whole tumor, and necrosis were manually segmented. Cerebral blood flow maps were calculated and mean perfusion values (MPV) within ET ROIs were extracted. Scatter plot of MPV and Kaplan-Meier graphs were generated using GraphPad Prism. Based on median survival of 420 days, patients were categorized into short (≤420 days) or long (>420 days) survival. A threshold ASL measured perfusion value of 30 mL/100g/min distinguished survival groups; hyper-perfusion correlated with shorter OS, while hypo-perfusion linked to longer OS. The mean±SD of PFS and OS for patients with shorter survival were 119±42 days and 230±117 days respectively, while for longer survival were 624±285 and 726±210 days respectively, and the values were statistically different (P<0.001for both). Kaplan-Meier curves showed significant OS and PFS differences (p<0.015, Hazard Ratio=3.056). Our data indicate that ASL measured baseline perfusion, after surgery and before chemoradiation treatment can benefit in predicting survival and treatment stratification in glioblastoma patients. ASL measured perfusion can be performed during radiation treatment and could provide valuable information that can be incorporated into emerging treatment strategies such as adaptive radiation. Citation Format: Durga Udayakumar, Limin Zhou, Marco Pinho, Yiming Wang, Ben Wagner, Michael Youssef, Joseph Maldjian, Ananth Madhuranthakam. Noninvasive magnetic resonance imaging measured quantitative perfusion as predictive and prognostic biomarker in glioblastoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 7163.