Messmann, H

ABSTRACT Objective Despite high stand‐alone performance, studies demonstrate that artificial intelligence (AI)‐supported endoscopic diagnostics often fall short in clinical applications due to human‐AI interaction factors. This video‐based trial on Barrett's esophagus aimed to investigate how examiner behavior, their levels of confidence, and system usability influence the diagnostic outcomes of AI‐assisted endoscopy. Methods The present analysis employed data from a multicenter randomized controlled tandem video trial involving 22 endoscopists with varying degrees of expertise. Participants were tasked with evaluating a set of 96 endoscopic videos of Barrett's esophagus in two distinct rounds, with and without AI assistance. Diagnostic confidence levels were recorded, and decision changes were categorized according to the AI prediction. Additional surveys assessed user experience and system usability ratings. Results AI assistance significantly increased examiner confidence levels ( p < 0.001) and accuracy. Withdrawing AI assistance decreased confidence ( p < 0.001), but not accuracy. Experts consistently reported higher confidence than non‐experts ( p < 0.001), regardless of performance. Despite improved confidence, correct AI guidance was disregarded in 16% of all cases, and 9% of initially correct diagnoses were changed to incorrect ones. Overreliance on AI, algorithm aversion, and uncertainty in AI predictions were identified as key factors influencing outcomes. The System Usability Scale questionnaire scores indicated good to excellent usability, with non‐experts scoring 73.5 and experts 85.6. Conclusions Our findings highlight the pivotal function of examiner behavior in AI‐assisted endoscopy. To fully realize the benefits of AI, implementing explainable AI, improving user interfaces, and providing targeted training are essential. Addressing these factors could enhance diagnostic accuracy and confidence in clinical practice.

ObjectivesAchalasia is an esophageal motility disorder that impairs quality of life and is often missed (20–50%) on endoscopy. A newly developed computer‐aided detection (CAD) software has shown high accuracy for achalasia diagnosis in preclinical settings. However, its benefit in a clinical setting remains unclear.MethodsBetween February and August 2023, 83 endoscopists from 27 centers assessed 50 randomized endoscopic videos (25 achalasia, 25 nonachalasia) without and with CAD. Endoscopists assessed videos without CAD, then with CAD after 2 months. The primary end‐point was improvement in sensitivity for nonexperienced endoscopists (no endoscopic experience of achalasia). Sensitivity, specificity, and accuracy with and without CAD were compared using the McNemar test.ResultsSensitivity for diagnosing achalasia increased significantly with CAD, rising from 74.2% (95% confidence interval [CI] 72.2–76.0%) to 91.2% (95% CI 89.9–92.4%) for all readers, showing a difference of 17.1% (95% CI 15.1–19.0%). Specifically, sensitivity improved from 66.9% (95% CI 63.6–70.0%) to 91.9% (95% CI 89.9–93.6%) among nonexperienced endoscopists, resulting in a difference of 25.0% (95% CI 21.7–28.4%), and from 79.5% (95% CI 77.1–81.8%) to 90.8% (95% CI 89.0–92.3%) among experienced endoscopists (endoscopic experience of at least one achalasia case), with a difference of 11.3% (95% CI 8.9–13.6%). Accuracy and specificity improved significantly with CAD assistance, regardless of reader's experience.ConclusionCAD improves achalasia detection by 17%, confirming preclinical results. The benefit was higher for nonexperienced endoscopists. CAD assistance may lead to prompt and effective treatment, minimizing the risk of false‐negative diagnosis in clinical practice.Trial registrationThis study was registered in the University Hospital Medical Information Network Clinical Trial Registry (https://www.umin.ac.jp/ctr/) number: UMIN000053047.