Cohen-Shelly, Michal

BACKGROUND: While clinical practice guidelines advocate for multidisciplinary heart team (MDHT) discussions in coronary revascularization, variability in implementation across health care settings remains a challenge. This variability could potentially be addressed by language learning models like ChatGPT, offering decision-making support in diverse health care environments. Our study aims to critically evaluate the concordance between recommendations made by MDHT and those generated by language learning models in coronary revascularization decision-making. METHODS: From March 2023 to July 2023, consecutive coronary angiography cases (n=86) that were referred for revascularization (either percutaneous or surgical) were analyzed using both ChatGPT-3.5 and ChatGPT-4. Case presentation formats included demographics, medical background, detailed description of angiographic findings, and SYNTAX score (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery; I and II), which were presented in 3 different formats. The recommendations of the models were compared with those of an MDHT. RESULTS: ChatGPT-4 showed high concordance with decisions made by the MDHT (accuracy 0.82, sensitivity 0.8, specificity 0.83, and kappa 0.59), while ChatGPT-3.5 (0.67, 0.27, 0.84, and 0.12, respectively) showed lower concordance. Entropy and Fleiss kappa of ChatGPT-4 were 0.09 and 0.9, respectively, indicating high reliability and repeatability. The best correlation between ChatGPT-4 and MDHT was achieved when clinical cases were presented in a detailed context. Specific subgroups of patients yielded high accuracy (>0.9) of ChatGPT-4, including those with left main disease, 3 vessel disease, and diabetic patients. CONCLUSIONS: The present study demonstrates that advanced language learning models like ChatGPT-4 may be able to predict clinical recommendations for coronary artery disease revascularization with reasonable accuracy, especially in specific patient groups, underscoring their potential role as a supportive tool in clinical decision-making.

Abstract Aims Some artificial intelligence models applied in medical practice require ongoing retraining, introduce unintended racial bias, or have variable performance among different subgroups of patients. We assessed the real-world performance of the artificial intelligence-enhanced electrocardiogram to detect left ventricular systolic dysfunction with respect to multiple patient and electrocardiogram variables to determine the algorithm’s long-term efficacy and potential bias in the absence of retraining. Methods and results Electrocardiograms acquired in 2019 at Mayo Clinic in Minnesota, Arizona, and Florida with an echocardiogram performed within 14 days were analyzed (n = 44 986 unique patients). The area under the curve (AUC) was calculated to evaluate performance of the algorithm among age groups, racial and ethnic groups, patient encounter location, electrocardiogram features, and over time. The artificial intelligence-enhanced electrocardiogram to detect left ventricular systolic dysfunction had an AUC of 0.903 for the total cohort. Time series analysis of the model validated its temporal stability. Areas under the curve were similar for all racial and ethnic groups (0.90–0.92) with minimal performance difference between sexes. Patients with a ‘normal sinus rhythm’ electrocardiogram (n = 37 047) exhibited an AUC of 0.91. All other electrocardiogram features had areas under the curve between 0.79 and 0.91, with the lowest performance occurring in the left bundle branch block group (0.79). Conclusion The artificial intelligence-enhanced electrocardiogram to detect left ventricular systolic dysfunction is stable over time in the absence of retraining and robust with respect to multiple variables including time, patient race, and electrocardiogram features.