Ramai, Daryl

Abstract Objectives To comprehensively evaluate the validity of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes for both prevalent diagnoses and less common diseases, and to assess the performance of a large language model (LLM)-based system in validating these codes. Materials and Methods This retrospective study analyzed hospital admissions from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. We developed a validated LLM-based system using GPT-4o, refined through iterative prompt engineering, to assess ICD-10-CM code validity. We measured the positive predictive value (PPV) of ICD-10-CM codes, PPV of principal and secondary diagnoses, and the performance of an LLM-based system in code validation. Results Among 865 079 assigned codes, the PPV was 84.6% (95% CI, 84.5%-84.6%). Principal diagnoses had a PPV of 93.9% (95% CI, 93.7%-94.1%), while secondary diagnoses had a PPV of 83.8% (95% CI, 83.7%-83.9%). The LLM system demonstrated high performance in validating ICD codes, achieving 93.6% accuracy, 95.4% sensitivity, and 85.2% specificity. Among correctly assigned secondary diagnoses, the majority (67.9%) represented historical or baseline conditions, while 32.1% reflected active conditions that deviated from baseline status; 22.3% of these emerged after hospital admission. PPV decreases with later diagnosis positions, with the largest decline occurring between principal and secondary diagnoses. Discussion and Conclusion In this large-scale evaluation, ICD-10-CM codes exhibited generally high accuracy, though variability existed by position and condition type. A validated LLM system performed comparably to physician review and offers a scalable means to improve coding accuracy. These findings support the potential for integrating LLM-based auditing into routine workflows to strengthen the quality of administrative and research data.

Abstract Artificial intelligence (AI) and mucosal exposure devices like Endocuff have independently improved the adenoma detection rate (ADR) during colonoscopy. This meta-analysis evaluated the combined effect of Endocuff and AI versus AI alone on colorectal neoplasia detection rates. We searched PubMed, Embase, and Web of Science for randomized controlled trials (RCTs) comparing the impact of Endocuff+AI versus AI alone on colorectal neoplasia detection. Primary outcome was ADR; secondary outcomes included advanced adenoma detection rate (AADR), sessile serrated lesion detection rate (SSLDR), cecal intubation time, and withdrawal time. Pooled risk ratios (RRs) and mean differences (MDs) with 95% confidence intervals (CIs) were calculated using a random-effects model. Three RCTs with 2404 subjects were included (n = 1198 Endocuff+AI vs. n = 1206 AI alone). ADR was significantly higher in the Endocuff+AI group than in the AI alone group (54% vs. 48%, respectively) (RR 1.12, 95% CI 1.03–1.21, P = 0.01, I2 = 0%). There was a trend toward higher AADR (12.3% vs. 10%, RR 1.23, 95% CI 0.96–1.59, P = 0.10, I2 = 17%) and SSLDR (17.6% vs. 15.5%, RR 1.16, 95% CI 0.96–1.40, P = 0.13, I2 = 0%) in the Endocuff+AI group compared with the AI alone group, but it did not reach statistical significance. Both cecal intubation time (MD -0.61 minutes; 95% CI -1.54–0.33; P = 0.20; I2 = 87%) and withdrawal time (MD -0.42 minutes; 95% CI -1.01–0.17; P = 0.17, I2 = 60%) were similar between the two groups. Endocuff combined with AI was superior to AI alone in improving the adenoma detection rate without increasing intubation or withdrawal times.