Osorio, Mariana Borras

PURPOSE Cancer recurrence is a critical outcome for patients and physicians. Retrospective cancer recurrence data can evaluate recurrence-directed treatment and generate novel interventions targeting recurrent cancers. However, large cancer databases do not provide recurrence-related information, stymying study at scale, and consequently require significant manual record review. Automated evaluation of records may allow for the rapid generation of easily analyzed data sets, accelerating the evaluation of recurrence altogether. METHODS Patients treated with radiation therapy at one tertiary referral center from 2010 to 2018 with a verified cancer status (cancer recurrence v no cancer recurrence) were identified. Patients with recurrent disease were initially identified through manual record review, and the associated pathology report was collected. Google Automated Machine Learning with Natural Language Processing (AutoNLP) and Google Gemini 1.5 Pro were used to generate a model for binary classification, with comparison to the gold-standard manually developed data set. RESULTS A total of 7,054 patients were identified. 3,431 (48.6%) were female, with a median age of 64 years. Head and neck (1,482, 21%), breast (1,480, 21%), upper GI (1,307, 18.5%), and lung/thorax (1,107, 15.7%) were the most common disease sites. Recurrence was verified for 1,546 patients (21.9%) using pathology reports, of which 1,249 positive cases were paired with 651 negative pathology reports for model development. Google Gemini 1.5 Pro consistently outperformed AutoNLP across all measurements of accuracy, generating a greater absolute difference in precision, recall, negative predictive value, and specificity, and a higher likelihood of correct classification at the individual level, rendering Gemini superior in recurrence status extraction. CONCLUSION AutoNLP and Google Gemini 1.5 Pro are promising tools for identifying recurrence from pathology reports, with the latter demonstrating superior overall performance, making it particularly suitable for clinical translation.

357 Background: Prostate cancer (PCa) is the most incident cancer in adult males in the United States, and the second highest in the world. CTCAE is the standardized method to grade the severity of treatment-related adverse events (AEs), but are tedious to collect and subject to inter-observer variability. This study aimed to evaluate the performance of LLMs in automatically extracting and grading CTCAE toxicities from clinical notes and patient-reported outcomes of PCa patients from a clinical trial. Methods: A total of 55 patients from NCT02874014 trial, undergoing proton therapy (PT) for localized PCa, were included. The ground truth of CTCAE graded toxicities was obtained from the trial’s results. Clinical notes within 3 days, and the most recent patient questionnaires within 90 days of toxicity assessment were retrieved from the electronic health record. A comprehensive prompt was developed to extract and CTCAE grade toxicities from the retrieved records. LLMs used for this analysis were OpenAI GPT-4o and Gemini 2.0 Flash. Phase 1 evaluated both LLMs' performance against the ground truth. To ensure the metrics reflected the models’ performance based only on the information available in the text provided, cases where both LLMs contradicted the ground truth were manually reviewed. Phase 2 recalculated LLMs' performance metrics using this adapted ground truth. A conservative stepwise ensemble model was developed to leverage the complementary strengths of both LLMs. Results: In the conservative stepwise ensemble model, Gemini 2.0 Flash served as the initial screening platform to identify AEs in step 1. All cases with identified toxicity events were then reviewed and confirmed using OpenAI GPT-4o in step 2. This effectively reduced false positive rates while preserving high sensitivity. The ensemble model achieved strong performance metrics: binary accuracy (0.954), sensitivity (0.961), specificity (0.953), F1 score (0.836), and grade accuracy (0.932). Conclusions: This study demonstrates the feasibility of the use of LLMs for extraction and CTCAE toxicity grading in PCa patients treated with RT. The ensemble model achieved strong performance metrics. Further validation is needed for other cancer diagnosis and treatment modalities. Phase Model Binary Accuracy Precision Sensitivity Specificity F1 Score Grade Accuracy 1 Gemini 0.946 0.672 1.000 0.939 0.804 0.919 1 OpenAI 0.940 0.785 0.637 0.978 0.703 0.895 2 Gemini 0.956 0.735 1.000 0.950 0.848 0.927 2 OpenAI 0.954 0.917 0.679 0.991 0.780 0.903

Background: Over 2.3 million liposuctions are performed annually with a complication rate of approximately 5%, including a death rate of 1 in 5000 because of blood loss. Artificial intelligence models offer potential for improving blood loss prediction and management in these procedures, analyzing extensive data to identify risk factors and accurately estimate blood loss. Methods: Data from 721 large-volume liposuction patients at 2 centers in Bogotá, Colombia, and Loja, Ecuador, between 2019 and 2023 were evaluated. Both centers followed identical perioperative protocols. The data set was split into training (621 patients) and testing (100 patients) sets. A supervised machine learning model was trained to predict blood loss. Model predictions were compared with clinical data using statistical validation metrics. Results: Most patients were women (79.2%), with median values of age of 37 years; weight, 65 kg; height, 165 cm; body mass index, 24.34 kg/m²; volemia, 3924.41 mL; infiltrated volume, 5800 mL; and aspirated volume, 3900 mL. Previous liposuction was noted in 32%. No significant differences were found between training and testing cohorts. The model achieved a mean absolute error of 22.09 mL, root mean square error of 34.13 mL, and an R ² value of 0.974, indicating high predictive accuracy and excellent model fit. Conclusions: The authors’ study has developed and validated an accurate artificial intelligence–based model to predict blood loss in large-volume liposuction, showing 94.1% accuracy. The authors’ model enhances preoperative planning and intraoperative management, potentially reducing complications and improving outcomes.