Riester, Kimberly

Abstract Large Language Models (LLM) are being widely adopted into the medical field for their impressive ability to analyze and summarize large amounts of text data. These models enable clinicians and researchers to extract meaningful insights from complex datasets and may assist with decision making. Here we present our workflows and application of LLM for the interpretation and summarization of clinical data related to CAR-T cell therapy. Using an LLM (Gemini 2.5 pro) in the Google Cloud Computing (GCP) environment, two applications were developed to analyze CAR-T cell therapy clinical data: 1) extracting and summarizing CRS and ICANS event-related data to streamline the compliance team workflow, and 2) identifying features available at time of CAR-T infusion able to classify patients into high- or low-monitoring needs 14 days post CAR-T. Patient data (vitals, labs, hematology notes, and EKGs) were extracted from the electronic medical record (EMR) using Google BigQuery into SQL tables in GCP. For each application, relevant data fields were retrieved, formatted into JSON objects, and embedded in the LLM prompt for context-aware processing. Both applications have undergone iterative prompt engineering after analyzing the LLM output against ground truth data in the EMR. For the application extracting and summarizing CRS and ICANS events, the LLM was optimized on Mayo Clinic Rochester data. When compared to the IEC compliance database, the LLM achieved 100% accuracy and F1 score for CRS events, and 96% accuracy and 82% F1 score for ICANS events. The match rate for CRS and ICANS grades were 83% and 89% respectively. We applied the same LLM to Mayo Clinic Arizona (MCA) and Mayo Clinic Florida (MCF) cases, who document clinical notes and toxicity flowsheet differently, and achieved an accuracy (MCA: 93%, MCF:93%) and F1 score (MCA: 96%, MCF: 96%) for CRS and accuracy (MCA: 81%, MCF:82%) and F1 score (MCA: 76%, MCF: 75%) for ICANS. LLM was able to capture events missed by manual reviews. For most of the discrepancies where compliance team final adjudication is needed, LLM will be updated to flag discrepancies for review by the compliance team. For the application related to the monitoring needs 14 days post CAR-T, the LLM identified 5 categories predictive of high or low monitoring needs post CAR-T infusion (disease status, inflammatory and tumor burden markers, hematologic status, renal function and performance status). Our model achieved a sensitivity of 85.7%, specificity of 23.8%, and F1 score of 65.5% for our first cohort, compared with data extracted by the LLM from the EMR. For our second cohort, with demographics statistically similar to cohort 1 and using the same 5 categories, the LLM achieved a sensitivity of 83.3%, a specificity of 28.6%, and F1 score of 65.4%. Our workflow and applications of LLM’s provide examples and guidance to others interested in applying LLM’s for clinical and research applications. Citation Format: Emmanuel Contreras Guzman, Matthew Jankowski, Andre De Menezes Silva Corraes, Malvika Gupta, Monica L. Shaw, Madiha Iqbal, Talal Hilal, Saurabh Chhabra, Ricardo Daniel Parrondo, Jody K. Mclean, Kim R. Riester, Kayla Joseph, Melinda Tan, Holly Ross, Cleyonia Barnett, Sylvia Carter, Semy Girmay, Rachel Wolan, Milana Ramsey, Christian Downhour, Kristy Morgan, Shae Sibley, Erica Rushing, Lucy Holmes, Allison Burgstahler, Stephen M. Ansell, Hassan Alkhateeb, Matthew Hathcock, Ramona Bruno, Allison C. Rosenthal, Hemant Murthy, Patrick B. Johnston, Jonas Paludo, Yi Lin. Applications of large language models to CAR-T cell therapy clinical data using Google cloud computing [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 1 (Regular Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(7 Suppl):Abstract nr 2740.

Abstract Introduction: CAR-T therapy has transformed cancer therapy. with its first Food and Drug Administration (FDA) approved product in August 2017. Given the expanding indications and growing experience, particularly with management of immediate toxicities requiring specialized expertise such as Cytokine Release Syndrome (CRS) and Immune Effector Cell-Associated Neurotoxicity syndrome (ICANS), FDA recently removed the Risk Evaluation and Mitigation Strategies (REMS) for CD19 and BCMA targeting CAR-T on June 26th2025. In particular, given that CRS and ICANS typically occur in the first 2 weeks post CAR-T infusion, the minimum required stay near CAR-T treatment center is reduced from one month to 2 weeks. While this could reduce treatment burden for patients (pts) who need to travel away from home to receive CAR-T, there is a need to update the transition of care from CAR-T center to pts' local oncology practice. Pts carry in their rate of recovery and have varying supportive care needs in the second half of the month post CAR-T infusion. To assist with care transition and planning, we examine the use of AI multimodal large language model (LLM) to predict patients' supportive care needs 2 weeks after CAR-T infusion based on the clinical data at the time of CAR-T evaluation, prior to leukapheresis. This timepoint was selected to also maximize travel, lodging and caregiver planning for pts and family as well. Methodology: Pts who were treated with FDA-approved CAR-T at Mayo Clinic Rochester between December 2022 and June 2025 were examined. Based on experiences with referral practices in the Midwest region, we defined patients with high monitoring needs as those who between day 14-30 post CAR-T had either hospital admission due to recurrent CRS or ICANS or required transfusion of more than 1 blood product per week (red blood cell or platelets). Others were classified as low monitoring needs. Clinical notes, labs, vital signs and electrocardiogram at the time of CAR-T evaluation were abstracted and summarized by Gemini LLM for association with patients with high versus low monitoring needs post CAR-T. Model was developed and tested for predicted needs and concordance with actual outcome of these pts. Results: Among the 362 pts who received FDA approved CAR-T in the timeframe, 84 pts had high monitoring needs. An additional 84 pts with low monitoring needs were randomly selected from 278 pts. These were divided into a training cohort and test cohort of 84 pts each, with 42 pts with high and 42 pts with low monitoring needs. Demographics of the two cohorts displayed no statistical difference. LLM analysis of the clinical data at the time of CAR-T eval identified 5 categories associated with risk for high monitoring needs: 1) Disease status (clinical notes about bulky, progressive disease or CNS involvement); 2) inflammatory and tumor burden markers (CRP, LDH); 3) hematologic status (platelet, plt; neutrophil count ANC); 4) renal function; 5) performance status. Using common clinical lab parameter convention (platelets <100, ANC< 1, CrCl< 30ml/min), and prioritizing for not missing pts with high monitoring needs, LLM model was defined to predict patients for high monitoring needs if high risk feature was identified in category 1 or any category combinations; whereas patients with no high risk feature or only 1 category that is not disease status was predicted as low monitoring needs. In this model, the sensitivity was 85.7%, specificity 23.8%, F1 score 65.5%. Applying this model to test cohort 2, the sensitivity was 83.3%, specificity 28.6%, and F1 score 65.4%. Discussion: We demonstrate proof-of-concept that LLM can be used to analyze clinical data at the time of evaluation for CAR-T eligibility to predict for supportive care needs post treatment. Not surprisingly, features identified by LLM to be associated with high monitoring needs are those previously reported by us and others at the time of CAR-T evaluation and or pre-lymphodepletion to be associated with risk for cytopenia, infection, hospitalization and non-relapse mortality. Based on the summary output generated by LLM, additional opportunities for classification methods were identified to improve specificity and balanced accuracy of the LLM model. This is a promising tool for further development to assist clinicians and patients to plan their treatment journey.