Waddle, Mark R.

LBA5012 Background: There is a critical evidence gap on the relative benefits and harms of proton therapy (PT) compared with photon-based intensity-modulated radiation therapy (IMRT) in prostate cancer. Herein we report early results from COMPPARE, a non-randomized comparative study of outcomes with PT and IMRT in prostate cancer. Methods: Between 2018-2022, 2524 patients from 51 facilities (28 PT and 23 IMRT) enrolled in COMPPARE, including 1501 treated with PT and 1023 treated with IMRT, and were followed prospectively. Treatment naïve patients with localized prostate cancer except those with very high-risk disease were eligible. Treatment was pragmatic with daily standard fractionation (1.8-2.1 Gy) and moderate hypofractionation (2.4-3.1 Gy) permitted. Use of rectal spacers and androgen deprivation were at physician discretion. Patient-reported quality of life data and treatment team-reported toxicity and disease control outcomes were collected at baseline and pre-specified time intervals via VisionTreeOptimalCare; treatment planning and delivery data were collected via the Advanced Treatment Consortium. We analyzed 1404 PT patients and 939 IMRT patients. We report the early primary endpoint of the study, a comparison of patient reported bowel urgency and bowel frequency at 2-years using the EPIC tool, and the secondary endpoints of 2-year CTCAE v5 ≥ grade 2 gastrointestinal toxicity outcomes and 3-year freedom from biochemical progression (FFBP) results reported by the study teams. Results: Study sample size was targeted at 1,500 PT patients and 1,000 IMRT patients, and median follow-up time was 4.0 years. The primary endpoint and secondary toxicity 2-year endpoints focused on the alternative hypothesis of PT superiority. All results include inverse probability of treatment weighting. Using generalized linear mixed effects models, we found there to be no significant difference in EPIC bowel urgency scores (p = 0.324) or bowel frequency scores (p = 0.514) between PT and IMRT cohorts over time up to the 2-year endpoint. For grade 2 gastrointestinal toxicity, the 2-year cumulative incidence was 5.2% for PT and 5.6% for IMRT. Gray’s Test for competing risks showed a hazard ratio (HR) of 0.91 [95% confidence interval (CI): 0.65-1.28, p = 0.6] for PT versus IMRT. The exploratory non-inferiority endpoint of 3-year FFBP was 98.0% for PT and 97.9% for IMRT, and Gray’s Test showed a HR of 0.95 (95% CI: 0.52-1.71, p = 0.90). Conclusions: Analysis of early results of COMPPARE show no significant differences between PT and IMRT in patient reported bowel frequency and urgency, grade ≥2 gastrointestinal toxicity, or FFBP. Assessment of long-term disease control, late toxicity and secondary malignancy will require longer follow-up. Clinical trial information: NCT03561220 .

PURPOSE Cancer recurrence is a critical outcome for patients and physicians. Retrospective cancer recurrence data can evaluate recurrence-directed treatment and generate novel interventions targeting recurrent cancers. However, large cancer databases do not provide recurrence-related information, stymying study at scale, and consequently require significant manual record review. Automated evaluation of records may allow for the rapid generation of easily analyzed data sets, accelerating the evaluation of recurrence altogether. METHODS Patients treated with radiation therapy at one tertiary referral center from 2010 to 2018 with a verified cancer status (cancer recurrence v no cancer recurrence) were identified. Patients with recurrent disease were initially identified through manual record review, and the associated pathology report was collected. Google Automated Machine Learning with Natural Language Processing (AutoNLP) and Google Gemini 1.5 Pro were used to generate a model for binary classification, with comparison to the gold-standard manually developed data set. RESULTS A total of 7,054 patients were identified. 3,431 (48.6%) were female, with a median age of 64 years. Head and neck (1,482, 21%), breast (1,480, 21%), upper GI (1,307, 18.5%), and lung/thorax (1,107, 15.7%) were the most common disease sites. Recurrence was verified for 1,546 patients (21.9%) using pathology reports, of which 1,249 positive cases were paired with 651 negative pathology reports for model development. Google Gemini 1.5 Pro consistently outperformed AutoNLP across all measurements of accuracy, generating a greater absolute difference in precision, recall, negative predictive value, and specificity, and a higher likelihood of correct classification at the individual level, rendering Gemini superior in recurrence status extraction. CONCLUSION AutoNLP and Google Gemini 1.5 Pro are promising tools for identifying recurrence from pathology reports, with the latter demonstrating superior overall performance, making it particularly suitable for clinical translation.

357 Background: Prostate cancer (PCa) is the most incident cancer in adult males in the United States, and the second highest in the world. CTCAE is the standardized method to grade the severity of treatment-related adverse events (AEs), but are tedious to collect and subject to inter-observer variability. This study aimed to evaluate the performance of LLMs in automatically extracting and grading CTCAE toxicities from clinical notes and patient-reported outcomes of PCa patients from a clinical trial. Methods: A total of 55 patients from NCT02874014 trial, undergoing proton therapy (PT) for localized PCa, were included. The ground truth of CTCAE graded toxicities was obtained from the trial’s results. Clinical notes within 3 days, and the most recent patient questionnaires within 90 days of toxicity assessment were retrieved from the electronic health record. A comprehensive prompt was developed to extract and CTCAE grade toxicities from the retrieved records. LLMs used for this analysis were OpenAI GPT-4o and Gemini 2.0 Flash. Phase 1 evaluated both LLMs' performance against the ground truth. To ensure the metrics reflected the models’ performance based only on the information available in the text provided, cases where both LLMs contradicted the ground truth were manually reviewed. Phase 2 recalculated LLMs' performance metrics using this adapted ground truth. A conservative stepwise ensemble model was developed to leverage the complementary strengths of both LLMs. Results: In the conservative stepwise ensemble model, Gemini 2.0 Flash served as the initial screening platform to identify AEs in step 1. All cases with identified toxicity events were then reviewed and confirmed using OpenAI GPT-4o in step 2. This effectively reduced false positive rates while preserving high sensitivity. The ensemble model achieved strong performance metrics: binary accuracy (0.954), sensitivity (0.961), specificity (0.953), F1 score (0.836), and grade accuracy (0.932). Conclusions: This study demonstrates the feasibility of the use of LLMs for extraction and CTCAE toxicity grading in PCa patients treated with RT. The ensemble model achieved strong performance metrics. Further validation is needed for other cancer diagnosis and treatment modalities. Phase Model Binary Accuracy Precision Sensitivity Specificity F1 Score Grade Accuracy 1 Gemini 0.946 0.672 1.000 0.939 0.804 0.919 1 OpenAI 0.940 0.785 0.637 0.978 0.703 0.895 2 Gemini 0.956 0.735 1.000 0.950 0.848 0.927 2 OpenAI 0.954 0.917 0.679 0.991 0.780 0.903

Abstract Background Partial breast irradiation (PBI) is an effective treatment for early-stage breast cancer. However, evidence comparing different PBI modalities is limited. Methods This trial emulation uses electronic health records from a multi-state large tertiary health system. Three PBI modalities were compared: proton radiation therapy, photon radiation therapy, and applicator-based brachytherapy. Inverse probability weighting (IPW) was used to balance confounders. The primary outcome was ipsilateral breast recurrence (IBR). Results Between 2013 and 2023, 1041 women with early-stage breast cancer were treated with PBI. Of these, 1026 patients were included in the analyses. The median age was 66 years; 22.51% had ductal carcinoma in situ (DCIS), and 97.56% were estrogen receptor positive. With median follow-up of 38.2 months, the rate of IBR in the overall group was 3.05% (95% confidence interval [CI] = 1.99% to 4.65%) at 3 years. After IPW, compared with photon PBI, the hazard ratio for IBR for proton PBI was 2.11 (95% CI = 0.70 to 6.33) and for brachytherapy, 3.86 (95% CI = 1.39 to 10.69). Compared with proton PBI, the hazard ratio for IBR for brachytherapy was 1.83 (95% CI = 0.88 to 3.81). IBR risk was similar across PBI modalities for patients with tumor size ≤ 10 mm and in patients treated for DCIS. Among patients with tumor size >10 mm, the hazard ratio for IBR for brachytherapy was 7.64 (95% CI = 1.64 to 35.58) and 3.59 (95% CI = 1.22 to 10.56) relative to photons and protons, respectively. Conclusion PBI with applicator-based brachytherapy was associated with higher IBR than photon or proton PBI in patients with tumor size > 10 mm, suggesting the need for more careful PBI patient selection for this modality.

Objectives: For locally advanced rectal adenocarcinoma (R-ACA), a nonoperative management (NOM) approach has emerged as a guideline-supported treatment option. However, the variables associated with NOM receipt are unknown. Methods: Utilizing the National Cancer Database, we performed a retrospective cohort study of adults with stage 1 to 3 R-ACA managed with curative intent from 2004 to 2018. The primary outcome was the proportion of patients receiving NOM versus surgery. The secondary outcome was survival among NOM patients. Results: A total of 128,297 patients were included. In all, 115,888 (90.3%) received surgery and 12,409 (9.7%) received NOM. Receipt of NOM was associated with age above 70, Charlson-Deyo score of 0, race (Black, Asian or Pacific Islander, or other vs. White), insurance status, geographical region, treatment in a community facility, year of diagnosis (2012-2018 vs. 2004-2011), tumor grade 1 versus ≥ 2, clinical T-stage ≥ 2, and clinical N1 or N2 versus N0. In the NOM cohort, poorer overall survival was associated with age 70 and above, male sex, Charlson-Deyo score ≥ 1, insurance status, geographical region, rural urban density versus metro/urban, treatment in a community facility, year of diagnosis (2004-2011 vs. 2012-2018), clinical T4 versus T1, clinical N1 or N2 versus N0, grade 3 versus 1, treatment with a radiotherapy dose <45 Gy versus 45 to 54 Gy, and omission of chemotherapy. Conclusions: Several demographic factors and social determinants of health were associated with receipt of NOM and overall survival. With the increasing utilization of NOM, it will be important to understand the drivers of treatment decisions and influences on access to the desired treatment approach.

Abstract Introduction While randomized clinical trials (RCT) confirmed superiority of Mepitel film (MF) in reducing acute radiation dermatitis (ARD) compared to standard-of-care (SoC), the incremental cost difference has limited its use. A cost-effectiveness analysis (CEA) was conducted from a Canadian healthcare payer’s perspective to guide policy decisions. Methods: A decision model was constructed to perform a CEA for MF compared to SoC (moisturisers) for prevention of grade 2 or higher ARD following adjuvant hypo-fractionated whole-breast radiotherapy (RT) based on a Canadian multicentre RCT. Direct and indirect cost data were collected from two oncology centres in Canada. Quality-of-life (QoL) utility values were derived from mapping Dermatology Life Quality Index (DLQI) scores for patients with grade 2 or higher ARD at week 6 of RT to EQ-5D. A willingness-to-pay (WTF) threshold of CAD 50,000 per quality-adjusted life years (QALY) gained was used. One-way sensitivity analysis was performed to account for uncertainty in decision model assumptions. Results: Base case analysis demonstrated that MF is cost-effective in preventing grade 2 or higher ARD as compared with SoC with an incremental cost-effectiveness ratio (ICER) of CAD 3,366 per QALY gained. When nursing costs were included, MF resulted in an ICER of CAD 2,823 per QALY gained. One-way sensitivity analysis showed that the results were most sensitive to the QoL utility value for ARD. All sensitivity variations remained well below the WTP threshold. Conclusions: MF is a cost-effective intervention for preventing high-grade ARD and should be recommended for patients with breast cancer undergoing adjuvant RT.

5108 Background: We previously proposed a hybrid framework combining LLMs and rule based algorithm (RBA) for automating risk stratification in locPCa. Herein, we aim to validate the risk stratification agent (RSA) in a prospective cohort, develop & evaluate the treatment recommendation agent (TRA) based on NCCN guidelines, and develop an interactive interface to facilitate clinicians for accurate risk stratification and treatment recommendations at the point of care. Methods: This study included pts with locPCa (2004-2024) presenting at Mayo Clinic with at least 1 positive prostate biopsy and MRI report available. For RSA prospective validation, GPT4 extracted key phenotypic variables (PSA, T stage, prostate volume, number of cores, Gleason patterns, grade group) from unstructured MRI and biopsy reports using a zeroshot prompt. An RBA then classified pts into NCCN risk groups. The agent performance was compared with the treating clinician's documentation and evaluated against gold-standard labels manually annotated by two independent clinicians. For development of TRA, two experiments were performed using GPT4 with and without retrieval-augmented generation (RAG) to generate treatment plans. Generated treatment plans were evaluated using NCCN guideline-informed treatment decision tree based algorithm (DTA). Evaluation metrics, weighted – accuracy (acc) and F1, were computed. A clinician facing interface (lisr.org/risk) was developed to provide accurate risk stratification and treatment plans. Results: A total of 858 pts were included (500 for prospective validation, 358 for treatment recommendations). Prospective validation for RSA demonstrated a higher F1 score of 0.89 compared to the treating clinician (F1: 0.58). Treatment recommendation experiments showed that GPT4 with RAG achieved higher acc (64% full – all correct treatment options and 36% partial – at least 1 correct treatment option) compared to GPT4 alone (35% and 65%, respectively). Sensitivity analysis using DTA-informed GPT4 note generation achieved 94% full acc and 6% partial acc. GPT4 alone generated hallucinated treatment options in 71% of cases, while GPT4 with RAG reduced this to 32%. Conclusions: The multi-agent framework based on LLM and RBA achieves high accuracy in risk stratification and treatment recommendation for locPCa. A multi-agent framework with an interactive interface holds high promise to enable efficient, accurate decision-making, and improve locPCa management at the point of care Performance evaluation. RSA(acc; F1) % TRA(full acc; partial acc) % Clinician RSA* GPT4 RAG+GPT4 DTA+GPT4 Overall 60; 58 89; 89 29; 71 64; 36 94; 6 V. High 94; 33 96; 96 55; 45 69; 31 93; 7 High 46; 52 92; 89 - - - Int. Unf 80; 61 91; 90 5; 95 38; 62 93; 7 Int. Fav 57; 64 84; 87 0; 100 98; 2 100; 0 Low 57; 65 87; 80 100; 0 95; 5 90; 10 V. Low 8; 13 50; 67^ - - - *Statistical significance (p<0.01) except for ^V. Low.

Objectives/Goals: Our study’s objective is to evaluate RadOnc-GPT, a GPT-4o powered LLM, in generating responses to in-basket messages related to prostate cancer treatment in the Radiation Oncology department. By integrating it with electronic health record (EHR) systems, the goal is to assess its impact on clinician workload, response quality, and efficiency in healthcare communication. Methods/Study Population: RadOnc-GPT was integrated with patient EHRs from both hospital-wide and radiation-oncology-specific databases. The study examined 158 pre-recorded in-basket message interactions from 90 non-metastatic prostate cancer patients. Quantitative natural language processing analysis and two randomized single-blinded grading studies, involving four clinicians and four nurses, were conducted to evaluate RadOnc-GPT’s response quality in completeness, correctness, clarity, empathy, and estimated editing time. Response times were measured to estimate the time saved for clinicians and nurses. The study population included patient messages across all phases of care (pre-, during, and post-treatment) for those undergoing radiotherapy. Results/Anticipated Results: In the single-blinded grader study, clinician graders evaluated 316 responses (158 from human care teams and 158 from RadOnc-GPT). Results showed RadOnc-GPT outperformed human responses in empathy and clarity, while humans excelled in completeness and correctness. Sentiment analyses using TextBlob and VADER revealed RadOnc-GPT responses had a positive mean score of 0.25, whereas human responses clustered around neutral. VADER analysis indicated a high median score for RadOnc-GPT, nearing 1.0, reflecting predominantly positive sentiment, while human responses displayed a broader sentiment range, indicating sensitivity to context. Clinicians averaged 3.60 minutes (SD 1.44) to respond, compared to 6.39 minutes (SD 4.05) for nurses, highlighting RadOnc-GPT’s efficiency in generating timely responses. Discussion/Significance of Impact: RadOnc-GPT effectively generated responses to individualized patient in-basket messages, comparable to those from radiation oncologists and nurses. While human oversight is still necessary to avoid errors, RadOnc-GPT can speed up response times and reduce pressure on care teams, shifting their role from drafting to reviewing responses.

425 Background: Prostate cancer (PC) ranks among the most prevalent cancers in men, with incidence rising due to factors like aging populations and enhanced screening techniques. Effective data extraction from clinical records is crucial for advancing research and improving patient care; however, traditional manual methods can be time-consuming and susceptible to mistakes. This study seeks to evaluate RadOnc-GPT against human annotation for extracting essential clinical information from prostate cancer clinical records. Methods: RadOnc-GPT is a retrieval augmented generation (RAG)-based chatbot with access to a variety of patient data including treatment details, radiology reports, and clinical notes. RadOnc-GPT uses Open AI’s gpt-4o model. Patients who received curative intent radiotherapy as initial treatment for PC in our institution between 2014 and 2023 were randomly selected for evaluation. Relevant Clinical factors were extracted by a human expert annotator on our on-site RedCap database. The extraction of the same variables was performed using RadOnc-GPT. A comparison between the ground truth (human annotation) and RadOnc-GPT, to evaluate the performance of the AI system, was carried out including accuracy, recall and F1 score. Results: A total of 382 PC patients were included in the cohort for data extraction. The clinical variables extracted were the treatment intent, NCCN risk category, T and N stage, Gleason score, radiation volume, and Androgen Deprivation therapy (ADT) use. The overall performance of RadOnc-GPT was excellent, with the highest accuracy achieved for treatment intent (99.7%), Gleason score (98.4%), and Treatment Volume (90.1-99.5%). Conclusions: RadOnc-GPT performed excellent at extracting relevant clinical details from the electronic medical record and radiation oncology information systems with dramatic reduction in time and cost compared with manual review. LLMs could be highly useful for large-scale data extraction in PC research, offering a promising solution for automating the accurate data extraction process. Correct (n) Total (n) Accuracy Recall F1 score RT with curative intent 381 382 99.7 99.9 99.9 Prostate Cancer Risk Category 345 382 90.3 90.4 95.1 Primary tumor staging (T) at diagnosis 368 382 96.3 96.3 97.9 Regional nodal status (N) at diagnosis 352 382 92.1 92.2 95.9 Gleason Score Primary Secondary Total 376372376 382382382 98.497.498.4 98.597.498.5 99.298.799.2 Treated Areas Prostate Pelvic Nodes Para-Aortic Nodes Distant Site 380344378372 382382382382 99.590.19997.4 99.490.199.197.4 99.79599.598.7 ADT Use with RT 365 382 95.5 95.7 97.8

424 Background: Advances in prostate cancer (PC) care have led to a 98% 10-year survival rate. Prostate Specific Antigen (PSA) is the primary method of follow-up after definitive radiotherapy (RT), with benign PSA ‘bounce’ occurring in up to 30% of patients, causing uncertainty and anxiety. Distinguishing between PSA bounce and biochemical recurrence (BCR) remains challenging. The Phoenix Criteria is the standard definition of BCR since 2006, defined as PSA nadir + 2 ng/dl, but does not consider patient specific information. Our study aims to explore factors associated with bounce and BCR and develop a predictive tool to differentiate between these two events. Methods: PC patients who received RT at our institution between 2006 and 2022 were identified, including only patients with a PSA increase during follow-up. Exclusion criteria included palliative RT, prostatectomy, or insufficient follow-up information. Patients’ clinical data was collected, and PSA increase was classified as bounce or BCR. Variables were compared using Kruskal Wallis or Chi-square. Multivariate logistic regression was performed in the subset of patients with absolute PSA value >2 (n= 581). Results: Of 1783 patients reviewed due to PSA rise after treatment, 694 met the inclusion criteria. There were 213 (30.7%) with PSA bounce and 481 (69.3%) with recurrence. Prior to meeting the Phoenix Criteria, 20 (4.2%) had biopsy-proven recurrence and 18 (3.7%) imaging-proven recurrence. The median delta PSA was 0.8 ng/mL for bounce vs 5 ng/mL for recurrence (p<0.001). The median time from treatment to event was 14.4 months for bounce vs 45.3 months for BCR (p<0.001). Recurrence was significantly associated (p<0.001) with older age at diagnosis, N1 stage, higher risk category, Gleason score, and higher baseline PSA. In the subgroup of patients without ADT, recurrence was associated with a lower median first PSA post-RT (1.9 vs. 2.4, p=0.046) and a lower median nadir before event (0.6 vs. 1.8, p<0.001). Multivariate logistic regression achieved an accuracy of 0.94, with a sensitivity of 0.83, specificity of 0.98, and an area under the curve (AUC) of 0.97. Conclusions: Our results highlight significant differences in clinical factors between bounce and recurrence in post-RT patients. The multivariate logistic regression differentiated the events with 94% accuracy for patients with a rising PSA value >2 subgroup. We highlight the possible improvement that could be accomplished by personalizing a PSA threshold for recurrence versus the Phoenix definition. This tool offers patients and providers a highly specific tool to differentiate between PSA recurrence and bounce, and may reduce stress and unnecessary testing, or help diagnosis recurrence faster. Next steps include prospective validation of the model to assess the impact on clinical practice costs, patient experience, and cancer outcomes.