Ramasamy, Gokul

Abstract Echocardiography is the most widely used cardiac imaging modality, yet artificial intelligence-enabled interpretation remains limited by the inability of existing models to integrate visual assessment, quantitative measurement, and clinical reasoning within a unified framework. Here we present EchoAtlas, the first autoregressive vision-language model developed for echocardiographic interpretation. Trained on over 12.9 million question-answer pairs derived from approximately 2 million echocardiogram videos, EchoAtlas achieves 0.966 accuracy on multiple-choice questions in our internal test set and establishes a new state-of-the-art on the public MIMIC-EchoQA benchmark (0.699 vs. 0.508 previously). EchoAtlas also provides accurate quantitative measurements, segment-level regional wall motion assessment, longitudinal comparison, and diagnostic reasoning across diverse question formats — capabilities not previously demonstrated in this domain. These results highlight the potential of autoregressive vision-language models as a foundation for interactive echocardiographic interpretation, representing an early step toward scalable, auditable artificial intelligence systems in cardiology practice.

Abstract Background Despite the high precision of deep learning models on internal tests on CT, their effectiveness often drops on external validation due to artifacts caused by patient motion and implants like metal or silicone that were not accounted for in the carefully curated training data. Potential wide categories of “unknown” anomalies within a CT exam makes training a supervised model for out‐of‐distribution (OOD) identification impractical, especially when considering unseen external data. Purpose To develop an artificial intelligence (AI) model to detect and identify anomalies/OOD data in abdominal–pelvis CT exams for the purpose of improving the performance of downstream applications. Methods Our proposed 2D and 3D generative architecture receives the third lumbar vertebra (L3) slice (slice‐level model) or all the slices from the series (series‐level model), generates a reconstruction and the secondary part of our architecture—anomaly score computation block, computes the anomalies pixels/voxels (slice‐level/series‐level) to identify anomalous L3‐slices/volumes (slice‐level/series‐level). We trained on data from over 2850 abdominal–pelvis CT volumes from adults over age 50 years collected throughout multiple Mayo Clinic campuses (60% female; mean age: 66.9, 92.4% non‐Hispanic White) and tested on a prospective test set of 544 CTs from July 2024 (47.3% female; mean age: 70.9, 94% non‐Hispanic White) as well as an external test set. Results We found that while traditional methods show moderate success, our generative models—Vector Quantized Variational Autoencoder (VQVAE) and Vision Transformer‐Masked Autoencoder (VIT‐MAE)—deliver excellent results with negligible false positives (FPs) and are also superior in identifying varied types of OOD samples. Prospective analysis showed the model was able to handle the under‐documentation of anomaly in radiology reports with 86.11% true positive (TP) rate. We also performed external validation using the publicly available AbdominalCT‐1k dataset, which contains 1062 CT scans compiled from several existing benchmark datasets. The model achieved a 75.26% TP rate, while the 24.7% FP rate was primarily triggered by anomalies located outside the body. Conclusions The proposed method can be leveraged to detect both intra‐ and interclass OOD data from abdominal CT images and can assess the quality of CT datasets to provide actionable insights. This workflow is particularly valuable for nonshareable healthcare collaborations, where it can be deployed as a service within local firewalls for automated dataset curation without prior knowledge about the OOD types. The implementation of the algorithm is available in the GitHub: https://github.com/gokul‐ramasamy/implant_detection.git .

Education-level or socioeconomic background of patients may dictate their ability to understand medical jargon. Inability to understand primary findings from a radiology report may lead to unnecessary anxiety among patients or missed follow up. We aim to meet this challenge by developing a patient-sensitive summarization model for radiology reports. We selected computed tomography (CT) exams of chest as a use-case and collected 7,000 studies from Mayo Clinic. Summarization model was built on top of the T5 large language model (LLM) as our experiments indicated that its text-to-text transfer architecture was suited for abstractive text summarization, resulting in a model with 0.77B trainable parameters. Noisy ground truth for model training was collected by prompting LLaMA-13B model. We recruited experts (board-certified radiologists) and laymen to manually evaluate model-generated summaries generated by model. Our model rarely missed information as marked by majority opinion of radiologists. Laymen indicated 63% improvement in their understanding by reading model-generated layman summaries. Comparison with zero-shot performance of ChatGPT indicated that the proposed model reduced the rate of hallucination by half and rate of missing important information by fivefold. The proposed model can generate reliable summaries for radiology reports understandable by patients with vastly different levels of medical knowledge.

Abstract Background The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. Methods We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. Results Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). Conclusions We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.

PURPOSE Selection of appropriate adjuvant therapy to ultimately reduce the risk of breast cancer (BC) recurrence is a challenge for medical oncologists. Several automated risk prediction models have been developed using retrospective clinical data and have evolved significantly over the years in terms of predictors of recurrence, data usage, and predictive techniques (statistical/machine learning [ML]). METHODS Following PRISMA guidelines, we performed a systematic literature review of the aforementioned statistical and ML models published between January 2008 and December 2022 through searching five digital databases—PubMed, ScienceDirect, Scopus, Cochrane, and Web of Science. The comprehensive search yielded a total of 163 papers and after a screening process focusing on papers that dealt exclusively with statistical/ML methods, only 23 papers were deemed appropriate for further analysis. We benchmarked the studies on the basis of development, evaluation metrics, and validation strategy with an added emphasis on racial diversity of patients included in the studies. RESULTS In total, 30.4% of the included studies use statistical techniques, while 69.6% are ML-based. Among these, traditional ML models (support vector machines, decision tree, logistic regression, and naïve Bayes) are the most frequently used (26.1%) along with deep learning (26.1%). Deep learning and ensemble learning provide the most accurate predictions (AUC = 0.94 each). CONCLUSION ML-based prediction models exhibit outstanding performance, yet their practical applicability might be hindered by limited interpretability and reduced generalization. Moreover, predictive models for BC recurrence often focus on limited variables related to tumor, treatment, molecular, and clinical features. Imbalanced classes and the lack of open-source data sets impede model development and validation. Furthermore, existing models predominantly overlook African and Middle Eastern populations, as they are trained and validated mainly on Caucasian and Asian patients.