Citations 1
Computer-Vision Approach to Triaging Patient-Submitted Photos of Intestinal Ostomies
Varghese, Chris, Choudhary, Ashok, Larson, E W, Lee, Frank, Baier, Michele, Glabe, CG, Just, Jora N., Friedrich, Stacy, Merchea, Amit, Brady, Justin, Selig, Jody L., Thiels, Cornelius A., Salehinejad, Hojjat , Perry, William R.G., Varghese, Chris , Varghese, Chris , Baier, Michele C, Glabe, Cody L, Friedrich, Stacy A, Perry, William R , Lee, Frank, Glabe, Cody L., Merchea, Amit , Brady, Justin T. , Thiels, Cornelius A. , Perry, William
Diseases of the Colon & Rectum | July 17, 2026 | Volume: No volume available | Issue: No issue available | Pages: No page available
BACKGROUND: Surgeons and ostomy nurses receive a high volume of stoma photos from patients. OBJECTIVE: This study aimed to develop and validate an automated and scalable artificial intelligence pipeline for identification and triage of patient-submitted intestinal ostomy photos. DESIGN: Retrospective derivation and validation study with stakeholder engagement to guide model design. Clinical teams reviewed photos in duplicate and categorized them into: (a) obstructed view, (b) healthy stoma, (c) suitable for conservative management, or (d) requiring in-person review. Pre-trained neural networks including MobileNetV4, ResNet50, ViT and CLIP-ViT were fine-tuned with 5-fold cross-validation. SETTINGS: Nine Mayo Clinic hospitals (2019–2022). PATIENTS: Adult patients (≥18 years) undergoing surgery who submitted images within 30 days after surgery. MAIN OUTCOME MEASURES: Model performance was evaluated with area under the receiver operator curve, precision, recall, and F1-scores. RESULTS: In this study, 538 photos of abdominal ostomies were sent in by 191 patients (median age 52 years, IQR 40–63; 60.4% female). Expert consensus triaged 236 (43.9%) photos as having an obstructed view, 30 (5.6%) as healthy, 177 (32.9%) for conservative management, and 95 (17.7%) for in-person review. All models achieved area under the receiver operator curve >0.97 for identifying stomas. The CLIP-ViT model performed best at triage (macro- area under the receiver operator curve 0.94 ± 0.06; F1 0.77 ± 0.10). End-to-end detection and triage using CLIP-ViT achieved area under the receiver operator curve of 0.99 ± 0.01, precision of 0.87 ± 0.08, recall of 0.93 ± 0.04, and F1 of 0.89 ± 0.07. Attention maps showed that models focused on stomas to determine classification. LIMITATIONS: Low sample size and lack of prospective validation. CONCLUSIONS: A vision language model pipeline accurately detected and triaged patient-submitted ostomy photos. Prospective evaluation is now needed to support integration into multidisciplinary digital workflows. See Video Abstract .