Banerjee, Imon

Abstract Echocardiography is the most widely used cardiac imaging modality, yet artificial intelligence-enabled interpretation remains limited by the inability of existing models to integrate visual assessment, quantitative measurement, and clinical reasoning within a unified framework. Here we present EchoAtlas, the first autoregressive vision-language model developed for echocardiographic interpretation. Trained on over 12.9 million question-answer pairs derived from approximately 2 million echocardiogram videos, EchoAtlas achieves 0.966 accuracy on multiple-choice questions in our internal test set and establishes a new state-of-the-art on the public MIMIC-EchoQA benchmark (0.699 vs. 0.508 previously). EchoAtlas also provides accurate quantitative measurements, segment-level regional wall motion assessment, longitudinal comparison, and diagnostic reasoning across diverse question formats — capabilities not previously demonstrated in this domain. These results highlight the potential of autoregressive vision-language models as a foundation for interactive echocardiographic interpretation, representing an early step toward scalable, auditable artificial intelligence systems in cardiology practice.

A comprehensive, machine-interpretable taxonomy of artificial intelligence (AI) performance metrics provides a shared vocabulary and structured foundation to facilitate rigorous, transparent, and responsible evaluation of medical AI systems.

Abstract Background Despite the high precision of deep learning models on internal tests on CT, their effectiveness often drops on external validation due to artifacts caused by patient motion and implants like metal or silicone that were not accounted for in the carefully curated training data. Potential wide categories of “unknown” anomalies within a CT exam makes training a supervised model for out‐of‐distribution (OOD) identification impractical, especially when considering unseen external data. Purpose To develop an artificial intelligence (AI) model to detect and identify anomalies/OOD data in abdominal–pelvis CT exams for the purpose of improving the performance of downstream applications. Methods Our proposed 2D and 3D generative architecture receives the third lumbar vertebra (L3) slice (slice‐level model) or all the slices from the series (series‐level model), generates a reconstruction and the secondary part of our architecture—anomaly score computation block, computes the anomalies pixels/voxels (slice‐level/series‐level) to identify anomalous L3‐slices/volumes (slice‐level/series‐level). We trained on data from over 2850 abdominal–pelvis CT volumes from adults over age 50 years collected throughout multiple Mayo Clinic campuses (60% female; mean age: 66.9, 92.4% non‐Hispanic White) and tested on a prospective test set of 544 CTs from July 2024 (47.3% female; mean age: 70.9, 94% non‐Hispanic White) as well as an external test set. Results We found that while traditional methods show moderate success, our generative models—Vector Quantized Variational Autoencoder (VQVAE) and Vision Transformer‐Masked Autoencoder (VIT‐MAE)—deliver excellent results with negligible false positives (FPs) and are also superior in identifying varied types of OOD samples. Prospective analysis showed the model was able to handle the under‐documentation of anomaly in radiology reports with 86.11% true positive (TP) rate. We also performed external validation using the publicly available AbdominalCT‐1k dataset, which contains 1062 CT scans compiled from several existing benchmark datasets. The model achieved a 75.26% TP rate, while the 24.7% FP rate was primarily triggered by anomalies located outside the body. Conclusions The proposed method can be leveraged to detect both intra‐ and interclass OOD data from abdominal CT images and can assess the quality of CT datasets to provide actionable insights. This workflow is particularly valuable for nonshareable healthcare collaborations, where it can be deployed as a service within local firewalls for automated dataset curation without prior knowledge about the OOD types. The implementation of the algorithm is available in the GitHub: https://github.com/gokul‐ramasamy/implant_detection.git .

Abstract Background and Aims Women are underdiagnosed and undertreated for cardiovascular disease (CVD). Automatic quantification of breast arterial calcification (BAC) on screening mammography can identify women at risk for CVD. This study aimed to determine whether artificial intelligence–based automatic quantification of BAC from screening mammograms predicts CVD and mortality beyond PREVENT scores in a large, racially diverse, multi-institutional population. Methods This retrospective cohort study included 123 762 women from two healthcare systems who had screening mammograms. Breast arterial calcification was quantified using a transformer-based neural network for segmentation. Breast arterial calcification severity was categorized as zero (0 mm2), mild (>0–10 mm2), moderate (>10–25 mm2), and severe (>25 mm2). Kaplan–Meier analysis, Cox proportional hazards, and Fine–Gray competing event models were used to examine the association between BAC and major adverse cardiovascular events (MACE). Results Breast arterial calcification was detected in 16.1% (internal cohort) and 20.6% (external cohort) of women and provided significant prognostic value incremental to the PREVENT score. In PREVENT adjusted models, a clear dose–response was observed. Compared with zero BAC, mild [internal: hazard ratio (HR) 1.32, 95% confidence interval (CI) 1.10–1.59; external: HR 1.28, 95% CI 1.17–1.39], moderate (internal: HR 1.75, 95% CI 1.23–2.50; external: HR 1.79, 95% CI 1.55–2.06), and severe BAC (internal: HR 3.29, 95% CI 2.15–5.05; external: HR 2.80, 95% CI 2.36–3.32) were all prognostic for any MACE. Each 1 mm2 increase in BAC conferred an additional 2%–3% risk for MACE (P < .001). Conclusions Automatically quantified BAC is an independent predictor of MACE and mortality, adding prognostic value to the PREVENT score. This approach may provide an opportunistic cardiovascular risk assessment during routine mammography screening without additional radiation exposure to guide earlier and more effective preventive care for women.

PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.