Citations

Abstract Large Language Models (LLM) are being widely adopted into the medical field for their impressive ability to analyze and summarize large amounts of text data. These models enable clinicians and researchers to extract meaningful insights from complex datasets and may assist with decision making. Here we present our workflows and application of LLM for the interpretation and summarization of clinical data related to CAR-T cell therapy. Using an LLM (Gemini 2.5 pro) in the Google Cloud Computing (GCP) environment, two applications were developed to analyze CAR-T cell therapy clinical data: 1) extracting and summarizing CRS and ICANS event-related data to streamline the compliance team workflow, and 2) identifying features available at time of CAR-T infusion able to classify patients into high- or low-monitoring needs 14 days post CAR-T. Patient data (vitals, labs, hematology notes, and EKGs) were extracted from the electronic medical record (EMR) using Google BigQuery into SQL tables in GCP. For each application, relevant data fields were retrieved, formatted into JSON objects, and embedded in the LLM prompt for context-aware processing. Both applications have undergone iterative prompt engineering after analyzing the LLM output against ground truth data in the EMR. For the application extracting and summarizing CRS and ICANS events, the LLM was optimized on Mayo Clinic Rochester data. When compared to the IEC compliance database, the LLM achieved 100% accuracy and F1 score for CRS events, and 96% accuracy and 82% F1 score for ICANS events. The match rate for CRS and ICANS grades were 83% and 89% respectively. We applied the same LLM to Mayo Clinic Arizona (MCA) and Mayo Clinic Florida (MCF) cases, who document clinical notes and toxicity flowsheet differently, and achieved an accuracy (MCA: 93%, MCF:93%) and F1 score (MCA: 96%, MCF: 96%) for CRS and accuracy (MCA: 81%, MCF:82%) and F1 score (MCA: 76%, MCF: 75%) for ICANS. LLM was able to capture events missed by manual reviews. For most of the discrepancies where compliance team final adjudication is needed, LLM will be updated to flag discrepancies for review by the compliance team. For the application related to the monitoring needs 14 days post CAR-T, the LLM identified 5 categories predictive of high or low monitoring needs post CAR-T infusion (disease status, inflammatory and tumor burden markers, hematologic status, renal function and performance status). Our model achieved a sensitivity of 85.7%, specificity of 23.8%, and F1 score of 65.5% for our first cohort, compared with data extracted by the LLM from the EMR. For our second cohort, with demographics statistically similar to cohort 1 and using the same 5 categories, the LLM achieved a sensitivity of 83.3%, a specificity of 28.6%, and F1 score of 65.4%. Our workflow and applications of LLM’s provide examples and guidance to others interested in applying LLM’s for clinical and research applications. Citation Format: Emmanuel Contreras Guzman, Matthew Jankowski, Andre De Menezes Silva Corraes, Malvika Gupta, Monica L. Shaw, Madiha Iqbal, Talal Hilal, Saurabh Chhabra, Ricardo Daniel Parrondo, Jody K. Mclean, Kim R. Riester, Kayla Joseph, Melinda Tan, Holly Ross, Cleyonia Barnett, Sylvia Carter, Semy Girmay, Rachel Wolan, Milana Ramsey, Christian Downhour, Kristy Morgan, Shae Sibley, Erica Rushing, Lucy Holmes, Allison Burgstahler, Stephen M. Ansell, Hassan Alkhateeb, Matthew Hathcock, Ramona Bruno, Allison C. Rosenthal, Hemant Murthy, Patrick B. Johnston, Jonas Paludo, Yi Lin. Applications of large language models to CAR-T cell therapy clinical data using Google cloud computing [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 1 (Regular Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(7 Suppl):Abstract nr 2740.

Abstract Introduction: Reporting patient outcomes by CAR-T treatment centers is part of the accreditation requirement under the FACT (Foundation for Accreditation of Cell Therapy) Immune Effector Cell (IEC) Program standard. Currently, this is a time-consuming and manual process, making it increasingly difficult to sustain with the growing CAR-T applications and patient volume. The objective of this study is to evaluate how a Large Language Model (LLM) can be used to automate the abstraction and summarization of medical record for CAR-T toxicities. Methods: Clinical data from patients treated with CAR-T between January 2018 and June 2025, including hematology clinical notes, lab results, flowsheets, vital signs, and medication administration record, were used as the source of patient information. The American Society of Transplant and Cell Therapy (ASTCT) CAR-T toxicity grading guideline for cytokine release syndrome (CRS) and immune effector cell associated neurologic syndrome (ICANS) were used to define these toxicities. With the large context window of the LLM (Gemini 2.5 pro), we created a prompt by embedding both the patient data and the ASTCT guidelines for CRS and ICANS, along with an outline of specific information we aimed to abstract from the data. Additionally, the LLM was instructed to provide contextual justification for the data it extracted from patient information. The LLM-generated output included event onset date, resolution date, maximum grade, date of maximum grade, and medications given for management. The CRS and ICANS outcome reports generated by the LLM were then compared to the IEC compliance program data. Any discrepancies identified were reviewed manually. Results: Among patients (pt) treated with CAR-T in the specified timeframe, 47 were selected as training cohort. This cohort included pt with toxicities events, no events, and a mix of events of varying grades. For the test cohort, 66 pts who received CAR-T from March to June 2025 were included. Demographics for the training and test cohorts are reflective of evolving clinical practice (training, test cohorts: median age 69, 65; male 55%, 74%; lymphoma/leukemia 66%, 48%; multiple myeloma 34%, 52%; CD19 CAR-T 46%, 50%). Compared to data from the IEC compliance program, the LLM-generated summaries for CRS events demonstrated an accuracy of 90%, precision 95%, sensitivity 100%, specificity 80%, F1 score 97%, and Matthew's correlation coefficient 87%. For ICANS event, the LLM achieved an accuracy of 91%, precision 92%, sensitivity 92%, specificity 90%, F1 score 92%, Matthew's correlation coefficient 83%. Maximum grade, for both CRS and ICANS, were identified with >90% accuracy. However, dates for CRS and ICANS were less accurate, with the lowest accuracy noted in identifying the event resolution date, 45% for CRS and 67% for ICANS. Medical records were reviewed for all discrepancies between the LLM and the IEC Compliance Program data. The most common discrepancy was a mismatch in event resolution date, determined to be due to differing criteria used by the compliance team and the LLM prompt. Of note, a few discrepancies were attributed to missed information by the IEC Compliance Team, which was correctly identified and categorised by the LLM. An updated LLM prompt (LLMv2) was developed and applied to the test cohort. In version 2, the prompt was revised to include clearer instructions on how to summarize CRS and ICANS events, as well as new instructions on additional data elements to be summarized. The performance of LLMv2 on this cohort for CRS event identification was 100% across all categories (accuracy, precision, sensitivity, specificity, F1 score, and Matthew's correlation coefficient). For ICANS the LLMv2 achieved accuracy of 96%, precision 69%, sensitivity 100%, specificity 93%, F1 score 82%, Matthew's correlation coefficient 80%. Accuracy in identifying event resolution date improved significantly from 64% to 99%. Conclusions: Our study demonstrates that LLMs can significantly reduce the time required to abstract and summarize CAR-T toxicities, while maintaining high accuracy. One limitation of our study is that it is based on data from a single institution. However, plans are already underway to evaluate the use of LLMs in a multi-institutional setting.