Citations

PURPOSE Automated curation of breast cancer treatment data with minimal human involvement could accelerate the collection of statewide and nationwide evidence for patient management and assessing the effectiveness of treatment pathways. The primary challenges are the complexity and inconsistency of structured clinical data streams and accurate extraction of this information from free-text clinical narratives. MATERIALS AND METHODS We proposed a hybrid two-phase information extraction framework that combined a Unified Medical Language System parser (phase-1) with a fine-tuned large language model (LLM; phase-2) to extract longitudinal treatment timelines from time-stamped clinical notes. Our framework was developed through end-to-end joint learning as a question-answering model, where the model was trained to simultaneously answer five questions, each corresponding to a specific treatment. RESULTS We fine-tuned and internally validated the model on 26,692 patients with breast cancer (diagnosed between 2013 and 2020) receiving treatment at Mayo Clinic and externally validated the model on 162 randomly selected patients from Stanford Healthcare. Zero-shot LLM (out-of-the-box) had high specificity but low sensitivity, indicating that although these frameworks are useful for generic language understanding, they are lacking in terms of targeted clinical tasks. The proposed model achieved 0.942 average AUROC on the internal and 0.924 on the external data, demonstrating only marginal drop in performance when evaluated on external. The proposed model also achieved better trade-off between sensitivity (average: 79.2%) and specificity (average: 76.2%) compared with rule-based (average sensitivity: 70.5%, average specificity: 68.1%) and structured codes (average sensitivity: 64.1%, average specificity: 83.5%). CONCLUSION The proposed framework can extract temporal information about cancer treatments from various time-stamped clinic notes, regardless of the setting of treatment administration (inpatient or outpatient) or time frame. To support the cancer research community for such data curation and longitudinal analysis, we have packaged the code as a docker image, which needs minimal system reconfiguration and shared with an open-source academic license.

428 Background: Prostate cancer management presents a significant healthcare burden, with the need to efficiently triage patients for treatment. Our objective is to leverage large language models to predict physician-recommended treatment plans from unstructured clinical notes. By accurately predicting treatment plans, we aim to risk stratify and triage patients effectively, thereby optimizing the allocation of physician resources. Methods: 448 unstructured initial urology consultation patient notes following first positive prostate cancer biopsy were identified. The recommended and final treatments received were manually annotated to establish ground truth labels (Table 1). The dataset was split 80:20 for training and testing, preprocessed to remove plan sections and formatted into question-answer (QA) format. A domain-specific large language model (LLM) inspired by GPT and a specialized tokenizer (PCa- LLM) for prostate cancer terminology were developed. QA models were built using the PCa-LLM and compared with those using GPT-2 as the backbone to predict recommended and final treatments. Results: For the physician-recommended treatment plans, our LLM (PCa-LLM) showed superior performance with higher AUROC scores for curative vs. non-curative treatments (0.78 vs. 0.65), chemo-hormonal vs. other non-curative treatments (0.89 vs. 0.65), and surveillance vs. all other treatments (0.72 vs. 0.70), while both models achieved the same high AUROC of 0.99 for chemo-hormonal vs. all other treatments. For final treatments, PCa-LLM demonstrated better AUROC for curative vs. non-curative treatments (0.77 vs. 0.74) and chemo-hormonal vs. other non-curative treatments (0.71 vs. 0.66), while GPT2 outperformed PCa-LLM for surveillance vs. all other treatments (0.78 vs. 0.70). Both models achieved an AUROC of 0.99 for chemo-hormonal vs. all other treatments. Conclusions: PCa-LLM accurately predicted most treatment categories better than GPT2, with higher AUROC scores, and can be utilized to triage prostate cancer patients using initial consultation notes. Task Physician -Recommended Treatment Plan Final Treatment Received Curative Prostatectomy/Radiation 228 230 Non-Curative Focal Therapy 22 15 Active Surveillance 40 45 Chemo-hormonal 30 30 Model Predictions (AUROC) GPT2 PCa-LLM GPT2 PCa-LLM Curative vs. non-curative 0.65 0.78 0.74 0.77 Chemohormonal vs. other non-curative 0.65 0.89 0.66 0.71 Chemohormonal vs. all other 0.99 0.99 0.99 0.99 Surveillance vs. other non-curative 0.64 0.60 0.67 0.59 Surveillance vs. all other 0.70 0.72 0.78 0.70

Cardiovascular diseases and cancer are the leading causes of morbidity and mortality in modern society. Expanding cancer therapies that have improved prognosis may also be associated with cardiotoxicity, and extended life span after survivorship is associated with the increasing prevalence of cardiovascular disease. As such, the field of cardio-oncology has been rapidly expanding, with an aim to identify cardiotoxicity and cardiac disease early in a patient who is receiving treatment for cancer or is in survivorship. Artificial intelligence is revolutionizing modern medicine with its ability to identify cardiac disease early. This article comprehensively reviews applications of artificial intelligence specifically applied to electrocardiograms, echocardiography, cardiac magnetic resonance imaging, and nuclear imaging to predict cardiac toxicity in the setting of cancer therapies, with a view to reduce early complications and cardiac side effects from cancer therapies such as chemotherapy, radiation therapy, or immunotherapy.

Background: Prior studies indicate a relationship between migraine and MACE. Here we assess whether brain white matter hyperintensities (WMH), which are a common imaging finding in patient with migraine, contribute to the risk of MACE above demographics and common risk factors for MACE. Methods: 60,454 patients, ages 18-89, with a ICD-9 or ICD-10 migraine diagnosis code in 2010 or later were identified from the Mayo Clinic electronic health record. Only patients who were seen for migraine in the Neurology Department were included. Patients with a migraine diagnosis who did not have MACE were included only if they had at least two visits at Mayo Clinic during five years. Only patients with sex and race information were included. The final cohort included 577 migraine patients with and 598 migraine patients without MACE. Presence of WMH was determined from radiology notes. Individuals without a brain MRI were assumed not to have WMH. A logistic regression model that included sex, race, known lifetime MACE risk factors (atrial fibrillation, diabetes, hypertension, hyperlipidemia, tobacco use) and WMH as independent variables to predict MACE outcome was fit. Results: Significant factors that increased the risk of MACE in individuals with migraine included being Black or African American (adjusted OR: 2.9, 95% CI: 1.24-6.82, p = 0.014), presence of atrial fibrillation (adj. OR: 1.63, 95% CI: 1.23-2.17, p < 0.001), diabetes (adj. OR: 1.34, 95% CI: 1.02-1.75, p = 0.036), hypertension (adj. OR: 1.9, 95% CI: 1.39-2.6, p < 0.001), tobacco use (adj. OR: 1.66, 95% CI: 1.29-2.14, p < 0.001), and the presence of WMH (adj. OR: 1.43, 95% CI: 1.1-1.87, p = 0.008). Hyperlipidemia showed a marginal association (adj. OR: 1.34, 95% CI: 0.99-1.81, p = 0.061), while other variables such as sex and other racial/ethnic groups did not significantly alter the risk of MACE outcome. Discussion: Results indicate that African American race and presence of WMH in addition to common comorbidities independently increase the risk of MACE outcome.

Introduction: Prior data suggest that migraine increases the risk of stroke. However, there is a lack of understanding whether specific migraine features like white matter hyperintensities (WMH) commonly found on brain magnetic resonance (MR) imaging in migraine patients and migraine medications might contribute to Major Adverse Cardiovascular Event (MACE) outcomes including ischemic and hemorrhagic stroke. We assess MACE risk estimation model stratified by event types using Mayo Clinic Electronic Health Record (EHR) data. Methods: Case and control sets, based on MACE occurrence among migraine patients at Mayo Clinic Neurology Department, were propensity matched on age, sex, and MACE risk-factors (atrial fibrillation, diabetes, hyperlipidemia, hypertension, tobacco use) to capture migraine-specific predictors ( Figure 1) . Migraine medications were categorized into 9 classes; Triptans, Ditans, Gepants, Ergots, calcitonin gen-related peptide targeting medications (CGRPs), Beta-blockers, Topiramate, Tricyclic antidepressants, serotonin-norepinephrine reuptake inhibitors (SNRIs). Demographics (age and sex), migraine characteristics (chronic migraine, migraine with aura), and presence of WMH on MR imaging were curated. To model non-linear relationship between predictors, two comparative Random Forest (RF) models were evaluated for MACE prediction; i) Model-1: using demographics, migraine characteristics and WMH, ii) Model-2: medications, demographics, migraine characteristics and WMH ( Table 1) . Results: Model-2 outperformed Model-1 indicating combined interaction between medications, WMH, and migraine characteristics for estimating MACE risk. Model-2 achieved AUC of 0.68 for overall MACE, but showed higher performance for ischemic stroke (AUC 0.76). We visualized the effects of input features on ischemic stroke risk for four individual patients using SHAP force plots ( Figure 2 ). The model often considered use of betablockers, chronic migraine, and presence of WMH as factors for increasing ischemic stroke risk. Discussion: Our experiments show that medications, migraine features, WMH and demographics are more effective estimators for ischemic risk stroke risk compared to overall MACE risk, indicating stronger predictive correlation between ischemic stroke and migraine related clinical features. Interestingly, the model inferred the clinical practice pattern of not prescribing Triptans (TRIPTAN=0) to the patients with high risk of stroke from the data itself.

Abstract Artificial intelligence (AI) models in medicine often fall short in real-world deployment due to inability to incorporate multiple data modalities in their decision-making process as clinicians do. Clinicians integrate evidence and signals from multiple data sources like radiology images, patient clinical status as recorded in electronic health records, consultations from fellow providers, and even subtle clues using the appearance of a patient, when making decisions about diagnosis or treatment. To bridge this gap, significant research effort has focused on building fusion models capable of harnessing multi-modal data for advanced decision making. We present a broad overview of the landscape of research in multimodal AI for radiology covering a wide variety of approaches from traditional fusion modelling to modern vision-language models. We provide analysis of comparative merits and drawbacks of each approach to assist future research and highlight ethical consideration in developing multimodal AI. In practice, the quality and quantity of available training data, availability of computational resources, and clinical application dictates which fusion method may be most suitable.