Banerjee, Imon

BACKGROUND: Accurate assessment of left ventricular outflow tract (LVOT) gradients is critical for hypertrophic cardiomyopathy management, yet Doppler-based measurements are technically demanding and require expertise. The objective of this work was to develop a multi-view deep learning model capable of classifying LVOT obstruction (>20 mm Hg) using routine 2-dimensional echocardiographic windows without reliance on Doppler imaging. METHODS: We trained and externally validated a cross-attention-based video-to-video fusion framework that integrated EchoPrime-derived video representations from 3 standard transthoracic echocardiographic views to classify LVOT gradients. RESULTS: Training was performed on a derivation cohort (N=1833) from a tertiary care system in the United States, with model performance evaluated on an internally held-out test set (N=275) and a Korean external validation cohort (N=46). Single-view baselines showed limited discrimination (external area under the receiver operating curves, 0.47–0.70). Conversely, the domain-specific foundational model (EchoPrime) achieved superior single-view performance (area under the receiver operating curves, 0.75–0.80 internal; 0.79–0.83 external), highlighting the importance of echo-specific pretraining and temporal modeling. The proposed multi-view fusion further enhanced predictive performance, with the late fusion model reaching an area under the receiver operating curve of 0.84 on the external cohort with significant population-shift. CONCLUSIONS: These results suggest LVOT physiology is encoded in routine 2-dimensional imaging and can be leveraged for clinically relevant gradient classification without Doppler input. The proposed artificial intelligence-guided strategy demonstrates substantial cost savings compared with the screen-all approach. By integrating complementary spatial-temporal information across multiple views, our approach generalizes robustly across populations and may enable real-time decision support, extend LVOT assessment to portable or resource-limited settings, and complement Doppler-based evaluation for longitudinal hypertrophic cardiomyopathy management.

PURPOSE To evaluate open-source large language models (LLMs) for extracting cancer-specific phenotypic data, benchmark their performance against GPT4 models, and assess the impact of fine-tuning with training data sizes. METHODS Open-source LLMs (Mistral, LLaMa, MAMBA, BioMistral) were evaluated in zero-/one-shot and fine-tuned setups against GPT4-turbo/GPT4o to extract the cancer presence, progression, response, and metastatic sites from radiology impressions of patients with solid tumors treated at Dana-Farber Cancer Institute. Performance metrics (accuracy, precision, recall, F1-score) were computed. McNemar's odds ratio (OR), measuring which model is more likely to be correct when they disagree, was computed with 95% CI. Statistical significance was assessed using the alpha of .000139. RESULTS This study included 2,623 patients (25,273 radiology impressions). In zero-/one-shot settings, GPT4-turbo/GPT4o outperformed open-source LLMs. However, fine-tuned open-source LLMs achieved higher F1-scores than GPT4 models. Compared with the best-performing GPT4 model, fine-tuned Mistral0.2-7.3B (OR, 0.27 [95% CI, 0.20 to 0.36]; P < .00001), Mistral0.3-7.3B (OR, 0.26 [95% CI, 0.19 to 0.36]; P < .00001), LLaMa2-6.7B (OR, 0.30 [95% CI, 0.22 to 0.40]; P < .00001), LLaMa3.1-8B (OR, 0.37 [95% CI, 0.28 to 0.48]; P < .00001), and MAMBA-2.8B (OR, 0.32 [95% CI, 0.24 to 0.42]; P < .00001) showed significantly better performance in ascertaining disease progression. Performance was consistently better for inferring overall response, any evidence of cancer, and sites of metastases, with no significant differences among fine-tuned open-source LLMs. Fine-tuning gains plateaued at 25% of training data (5,718 impressions) and remained comparable at 5% (1,144 impressions). CONCLUSION Open-source LLMs, when fine-tuned using labeled data, can effectively automate the ascertainment of key radiophenotypic variables using only the impression section of radiology reports, without the full report text. Their consistent performance in small training sets suggests that these models may provide a scalable approach for phenotypic characterization of patients with cancer in real-world clinical settings.

Abstract Background Artificial intelligence (AI)–based, mammography breast cancer risk prediction models show improved discriminatory accuracy relative to clinical risk models. However, data on their calibration are limited. This study compared model performance of 3 clinical breast cancer risk models: Gail, Tyrer–Cuzick v8, and Breast Cancer Surveillance Consortium v3 with the MIRAI AI-risk model. Methods Digital mammograms were ascertained from a screening mammography cohort of 12 308 women within the Mayo Clinic Biobank with 250 incident breast cancers (176 invasive) within 5 years. We predicted 5-year breast cancer risk, estimated discriminatory accuracy (C-index), and calibration (observed-to-expected ratio) of both overall and invasive breast cancer and compared estimates using bootstrapping approaches. Results MIRAI demonstrated similar or improved discriminatory accuracy of overall breast cancer (C-index = 0.71, 95% confidence interval [CI] = 0.68 to 0.74) and invasive breast cancer (C-index = 0.71, 95% CI = 0.67 to 0.75) compared with clinical models (overall breast cancer: C-index = 0.59-0.68; invasive breast cancer: C-index = 0.60-0.68). MIRAI’s calibration for risk of overall breast cancer (observed-to-expected ratio = 0.96, 95% CI = 0.85 to 1.08) was improved compared with Gail (observed-to-expected ratio = 1.22, 95% CI = 1.07 to 1.38) and Breast Cancer Surveillance Consortium (observed-to-expected ratio = 1.38, 95% CI = 1.22 to 1.56) but similar to TC with volumetric percent density and polygenic risk score (observed-to-expected ratio = 0.99, 95% CI = 0.87 to 1.13). However, for low-risk women (approximately 50%), MIRAI overestimated risk of overall breast cancer. MIRAI also overestimated risk of invasive breast cancer across the risk spectrum (observed-to-expected ratio = 0.68, 95% CI = 0.58 to 0.78), whereas clinical models had good calibration (observed-to-expected ratio = 0.86–0.99). Conclusion MIRAI demonstrated stronger discriminatory accuracy than clinical models for 5-year overall and invasive breast cancer risk prediction but overestimated risk for both breast cancer endpoints. AI-based risk models should consider discriminatory accuracy and calibration for invasive cancer before implementation.

Abstract Background and Aims Women are underdiagnosed and undertreated for cardiovascular disease (CVD). Automatic quantification of breast arterial calcification (BAC) on screening mammography can identify women at risk for CVD. This study aimed to determine whether artificial intelligence–based automatic quantification of BAC from screening mammograms predicts CVD and mortality beyond PREVENT scores in a large, racially diverse, multi-institutional population. Methods This retrospective cohort study included 123 762 women from two healthcare systems who had screening mammograms. Breast arterial calcification was quantified using a transformer-based neural network for segmentation. Breast arterial calcification severity was categorized as zero (0 mm2), mild (&gt;0–10 mm2), moderate (&gt;10–25 mm2), and severe (&gt;25 mm2). Kaplan–Meier analysis, Cox proportional hazards, and Fine–Gray competing event models were used to examine the association between BAC and major adverse cardiovascular events (MACE). Results Breast arterial calcification was detected in 16.1% (internal cohort) and 20.6% (external cohort) of women and provided significant prognostic value incremental to the PREVENT score. In PREVENT adjusted models, a clear dose–response was observed. Compared with zero BAC, mild [internal: hazard ratio (HR) 1.32, 95% confidence interval (CI) 1.10–1.59; external: HR 1.28, 95% CI 1.17–1.39], moderate (internal: HR 1.75, 95% CI 1.23–2.50; external: HR 1.79, 95% CI 1.55–2.06), and severe BAC (internal: HR 3.29, 95% CI 2.15–5.05; external: HR 2.80, 95% CI 2.36–3.32) were all prognostic for any MACE. Each 1 mm2 increase in BAC conferred an additional 2%–3% risk for MACE (P &lt; .001). Conclusions Automatically quantified BAC is an independent predictor of MACE and mortality, adding prognostic value to the PREVENT score. This approach may provide an opportunistic cardiovascular risk assessment during routine mammography screening without additional radiation exposure to guide earlier and more effective preventive care for women.

PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.

Abstract Artificial intelligence (AI) models in medicine often fall short in real-world deployment due to inability to incorporate multiple data modalities in their decision-making process as clinicians do. Clinicians integrate evidence and signals from multiple data sources like radiology images, patient clinical status as recorded in electronic health records, consultations from fellow providers, and even subtle clues using the appearance of a patient, when making decisions about diagnosis or treatment. To bridge this gap, significant research effort has focused on building fusion models capable of harnessing multi-modal data for advanced decision making. We present a broad overview of the landscape of research in multimodal AI for radiology covering a wide variety of approaches from traditional fusion modelling to modern vision-language models. We provide analysis of comparative merits and drawbacks of each approach to assist future research and highlight ethical consideration in developing multimodal AI. In practice, the quality and quantity of available training data, availability of computational resources, and clinical application dictates which fusion method may be most suitable.

Education-level or socioeconomic background of patients may dictate their ability to understand medical jargon. Inability to understand primary findings from a radiology report may lead to unnecessary anxiety among patients or missed follow up. We aim to meet this challenge by developing a patient-sensitive summarization model for radiology reports. We selected computed tomography (CT) exams of chest as a use-case and collected 7,000 studies from Mayo Clinic. Summarization model was built on top of the T5 large language model (LLM) as our experiments indicated that its text-to-text transfer architecture was suited for abstractive text summarization, resulting in a model with 0.77B trainable parameters. Noisy ground truth for model training was collected by prompting LLaMA-13B model. We recruited experts (board-certified radiologists) and laymen to manually evaluate model-generated summaries generated by model. Our model rarely missed information as marked by majority opinion of radiologists. Laymen indicated 63% improvement in their understanding by reading model-generated layman summaries. Comparison with zero-shot performance of ChatGPT indicated that the proposed model reduced the rate of hallucination by half and rate of missing important information by fivefold. The proposed model can generate reliable summaries for radiology reports understandable by patients with vastly different levels of medical knowledge.