Robbins, Steven A.

INTRODUCTION: High-quality colonoscopy requires accurate risk stratification for surveillance per the 2020 US Multi-Society Task Force guidelines and adherence to 2024 American College of Gastroenterology/American Society for Gastrointestinal Endoscopy quality benchmarks. Both are operationally challenging in clinical practice. We developed and validated Colon-Pilot, a large language model—powered clinical decision support system using GPT-4o to automate and standardize both functions. METHODS: The system was evaluated in 2 operational modes: (i) a human-in-the-loop clinical decision support validation of surveillance recommendations for 596 colonoscopies, comparing concordance with 2020 US Multi-Society Task Force guidelines against expert consensus, and (ii) an automated administrative audit applying Colon-Pilot to 42,632 colonoscopies across the Mayo Clinic Health System to calculate 2024 American College of Gastroenterology/American Society for Gastrointestinal Endoscopy priority quality indicators. Recommendations were autogenerated unless predefined safety criteria triggered manual review. RESULTS: Colon-Pilot issued recommendations for 522 of 596 cases (87.6%) and flagged 12.4% for manual review. For automated cases, guideline-concordant accuracy was 97.5% (Cohen κ = 0.970) versus 69.7% (κ = 0.781) for original endoscopist recommendations. Discordant artificial intelligence (AI) cases (n = 13) most often recommended longer-than-appropriate intervals (62%). Applied to the enterprise data set, Colon-Pilot calculated performance exceeding 2024 targets: adenoma detection rate 49.8% (≥35%), sessile serrated lesion detection rate 17.7% (≥6%), bowel preparation adequacy 91.8% (≥90%), and cecal intubation rate 97.4% (≥95%). DISCUSSION: Colon-Pilot demonstrated high fidelity in applying surveillance guidelines and automated quality benchmarking, outperforming unassisted endoscopists in guideline adherence. By combining safety protocols with large-scale automated reporting, it offers a scalable solution for improving both efficiency and quality in colorectal cancer prevention.