Citations

5108 Background: We previously proposed a hybrid framework combining LLMs and rule based algorithm (RBA) for automating risk stratification in locPCa. Herein, we aim to validate the risk stratification agent (RSA) in a prospective cohort, develop & evaluate the treatment recommendation agent (TRA) based on NCCN guidelines, and develop an interactive interface to facilitate clinicians for accurate risk stratification and treatment recommendations at the point of care. Methods: This study included pts with locPCa (2004-2024) presenting at Mayo Clinic with at least 1 positive prostate biopsy and MRI report available. For RSA prospective validation, GPT4 extracted key phenotypic variables (PSA, T stage, prostate volume, number of cores, Gleason patterns, grade group) from unstructured MRI and biopsy reports using a zeroshot prompt. An RBA then classified pts into NCCN risk groups. The agent performance was compared with the treating clinician's documentation and evaluated against gold-standard labels manually annotated by two independent clinicians. For development of TRA, two experiments were performed using GPT4 with and without retrieval-augmented generation (RAG) to generate treatment plans. Generated treatment plans were evaluated using NCCN guideline-informed treatment decision tree based algorithm (DTA). Evaluation metrics, weighted – accuracy (acc) and F1, were computed. A clinician facing interface (lisr.org/risk) was developed to provide accurate risk stratification and treatment plans. Results: A total of 858 pts were included (500 for prospective validation, 358 for treatment recommendations). Prospective validation for RSA demonstrated a higher F1 score of 0.89 compared to the treating clinician (F1: 0.58). Treatment recommendation experiments showed that GPT4 with RAG achieved higher acc (64% full – all correct treatment options and 36% partial – at least 1 correct treatment option) compared to GPT4 alone (35% and 65%, respectively). Sensitivity analysis using DTA-informed GPT4 note generation achieved 94% full acc and 6% partial acc. GPT4 alone generated hallucinated treatment options in 71% of cases, while GPT4 with RAG reduced this to 32%. Conclusions: The multi-agent framework based on LLM and RBA achieves high accuracy in risk stratification and treatment recommendation for locPCa. A multi-agent framework with an interactive interface holds high promise to enable efficient, accurate decision-making, and improve locPCa management at the point of care Performance evaluation. RSA(acc; F1) % TRA(full acc; partial acc) % Clinician RSA* GPT4 RAG+GPT4 DTA+GPT4 Overall 60; 58 89; 89 29; 71 64; 36 94; 6 V. High 94; 33 96; 96 55; 45 69; 31 93; 7 High 46; 52 92; 89 - - - Int. Unf 80; 61 91; 90 5; 95 38; 62 93; 7 Int. Fav 57; 64 84; 87 0; 100 98; 2 100; 0 Low 57; 65 87; 80 100; 0 95; 5 90; 10 V. Low 8; 13 50; 67^ - - - *Statistical significance (p<0.01) except for ^V. Low.

Objectives/Goals: Our study’s objective is to evaluate RadOnc-GPT, a GPT-4o powered LLM, in generating responses to in-basket messages related to prostate cancer treatment in the Radiation Oncology department. By integrating it with electronic health record (EHR) systems, the goal is to assess its impact on clinician workload, response quality, and efficiency in healthcare communication. Methods/Study Population: RadOnc-GPT was integrated with patient EHRs from both hospital-wide and radiation-oncology-specific databases. The study examined 158 pre-recorded in-basket message interactions from 90 non-metastatic prostate cancer patients. Quantitative natural language processing analysis and two randomized single-blinded grading studies, involving four clinicians and four nurses, were conducted to evaluate RadOnc-GPT’s response quality in completeness, correctness, clarity, empathy, and estimated editing time. Response times were measured to estimate the time saved for clinicians and nurses. The study population included patient messages across all phases of care (pre-, during, and post-treatment) for those undergoing radiotherapy. Results/Anticipated Results: In the single-blinded grader study, clinician graders evaluated 316 responses (158 from human care teams and 158 from RadOnc-GPT). Results showed RadOnc-GPT outperformed human responses in empathy and clarity, while humans excelled in completeness and correctness. Sentiment analyses using TextBlob and VADER revealed RadOnc-GPT responses had a positive mean score of 0.25, whereas human responses clustered around neutral. VADER analysis indicated a high median score for RadOnc-GPT, nearing 1.0, reflecting predominantly positive sentiment, while human responses displayed a broader sentiment range, indicating sensitivity to context. Clinicians averaged 3.60 minutes (SD 1.44) to respond, compared to 6.39 minutes (SD 4.05) for nurses, highlighting RadOnc-GPT’s efficiency in generating timely responses. Discussion/Significance of Impact: RadOnc-GPT effectively generated responses to individualized patient in-basket messages, comparable to those from radiation oncologists and nurses. While human oversight is still necessary to avoid errors, RadOnc-GPT can speed up response times and reduce pressure on care teams, shifting their role from drafting to reviewing responses.

425 Background: Prostate cancer (PC) ranks among the most prevalent cancers in men, with incidence rising due to factors like aging populations and enhanced screening techniques. Effective data extraction from clinical records is crucial for advancing research and improving patient care; however, traditional manual methods can be time-consuming and susceptible to mistakes. This study seeks to evaluate RadOnc-GPT against human annotation for extracting essential clinical information from prostate cancer clinical records. Methods: RadOnc-GPT is a retrieval augmented generation (RAG)-based chatbot with access to a variety of patient data including treatment details, radiology reports, and clinical notes. RadOnc-GPT uses Open AI’s gpt-4o model. Patients who received curative intent radiotherapy as initial treatment for PC in our institution between 2014 and 2023 were randomly selected for evaluation. Relevant Clinical factors were extracted by a human expert annotator on our on-site RedCap database. The extraction of the same variables was performed using RadOnc-GPT. A comparison between the ground truth (human annotation) and RadOnc-GPT, to evaluate the performance of the AI system, was carried out including accuracy, recall and F1 score. Results: A total of 382 PC patients were included in the cohort for data extraction. The clinical variables extracted were the treatment intent, NCCN risk category, T and N stage, Gleason score, radiation volume, and Androgen Deprivation therapy (ADT) use. The overall performance of RadOnc-GPT was excellent, with the highest accuracy achieved for treatment intent (99.7%), Gleason score (98.4%), and Treatment Volume (90.1-99.5%). Conclusions: RadOnc-GPT performed excellent at extracting relevant clinical details from the electronic medical record and radiation oncology information systems with dramatic reduction in time and cost compared with manual review. LLMs could be highly useful for large-scale data extraction in PC research, offering a promising solution for automating the accurate data extraction process. Correct (n) Total (n) Accuracy Recall F1 score RT with curative intent 381 382 99.7 99.9 99.9 Prostate Cancer Risk Category 345 382 90.3 90.4 95.1 Primary tumor staging (T) at diagnosis 368 382 96.3 96.3 97.9 Regional nodal status (N) at diagnosis 352 382 92.1 92.2 95.9 Gleason Score Primary Secondary Total 376372376 382382382 98.497.498.4 98.597.498.5 99.298.799.2 Treated Areas Prostate Pelvic Nodes Para-Aortic Nodes Distant Site 380344378372 382382382382 99.590.19997.4 99.490.199.197.4 99.79599.598.7 ADT Use with RT 365 382 95.5 95.7 97.8

424 Background: Advances in prostate cancer (PC) care have led to a 98% 10-year survival rate. Prostate Specific Antigen (PSA) is the primary method of follow-up after definitive radiotherapy (RT), with benign PSA ‘bounce’ occurring in up to 30% of patients, causing uncertainty and anxiety. Distinguishing between PSA bounce and biochemical recurrence (BCR) remains challenging. The Phoenix Criteria is the standard definition of BCR since 2006, defined as PSA nadir + 2 ng/dl, but does not consider patient specific information. Our study aims to explore factors associated with bounce and BCR and develop a predictive tool to differentiate between these two events. Methods: PC patients who received RT at our institution between 2006 and 2022 were identified, including only patients with a PSA increase during follow-up. Exclusion criteria included palliative RT, prostatectomy, or insufficient follow-up information. Patients’ clinical data was collected, and PSA increase was classified as bounce or BCR. Variables were compared using Kruskal Wallis or Chi-square. Multivariate logistic regression was performed in the subset of patients with absolute PSA value >2 (n= 581). Results: Of 1783 patients reviewed due to PSA rise after treatment, 694 met the inclusion criteria. There were 213 (30.7%) with PSA bounce and 481 (69.3%) with recurrence. Prior to meeting the Phoenix Criteria, 20 (4.2%) had biopsy-proven recurrence and 18 (3.7%) imaging-proven recurrence. The median delta PSA was 0.8 ng/mL for bounce vs 5 ng/mL for recurrence (p<0.001). The median time from treatment to event was 14.4 months for bounce vs 45.3 months for BCR (p<0.001). Recurrence was significantly associated (p<0.001) with older age at diagnosis, N1 stage, higher risk category, Gleason score, and higher baseline PSA. In the subgroup of patients without ADT, recurrence was associated with a lower median first PSA post-RT (1.9 vs. 2.4, p=0.046) and a lower median nadir before event (0.6 vs. 1.8, p<0.001). Multivariate logistic regression achieved an accuracy of 0.94, with a sensitivity of 0.83, specificity of 0.98, and an area under the curve (AUC) of 0.97. Conclusions: Our results highlight significant differences in clinical factors between bounce and recurrence in post-RT patients. The multivariate logistic regression differentiated the events with 94% accuracy for patients with a rising PSA value >2 subgroup. We highlight the possible improvement that could be accomplished by personalizing a PSA threshold for recurrence versus the Phoenix definition. This tool offers patients and providers a highly specific tool to differentiate between PSA recurrence and bounce, and may reduce stress and unnecessary testing, or help diagnosis recurrence faster. Next steps include prospective validation of the model to assess the impact on clinical practice costs, patient experience, and cancer outcomes.

PURPOSE The COVID-19 pandemic led to rapid expansion of telemedicine. The implications of telemedicine have not been rigorously studied in radiation oncology, a procedural specialty. This study aimed to evaluate the characteristics of in-person patients (IPPs) and virtual patients (VPs) who presented to a large cancer center before and during the pandemic and to understand variables affecting likelihood of receiving radiotherapy (yield) at our institution. METHODS A total of 17,915 patients presenting for new consultation between 2019 and 2021 were included, stratified by prepandemic and pandemic periods starting March 24, 2020. Telemedicine visits included video and telephone calls. Area deprivation indices (ADIs) were also compared. RESULTS The overall population was 56% male and 93% White with mean age of 63 years. During the pandemic, VPs accounted for 21% of visits, were on average younger than their in-person (IP) counterparts (63.3 years IP v 62.4 VP), and lived further away from clinic (215 miles IP v 402 VP). Among treated VPs, living closer to clinic was associated with higher yield (odds ratio [OR], 0.95; P < .001). This was also seen among IPPs who received treatment (OR, 0.96; P < .001); however, the average distance from clinic was significantly lower for IPPs than VPs (205 miles IP v 349 VP). Specialized radiotherapy (proton and brachytherapy) was used more in VPs. IPPs had higher ADI than VPs. Among VPs, those treated had higher ADI ( P < .001). CONCLUSION Patient characteristics and yield were significantly different between IPPs and VPs. Telemedicine increased reach to patients further away from clinic, including from rural or health care–deprived areas, allowing access to specialized radiation oncology care. Telemedicine has the potential to increase the reach of other technical and procedural specialties.

Abstract Background This study aimed to determine whether proton craniospinal irradiation (CSI) decreased the dose to normal tissue and resulted in less toxicity than photon CSI for adult patients. Methods This single-institution retrospective analyzed differences in radiation doses, acute toxicity, and cost between proton and CSI for adult medulloblastoma patients. Results Of 39 total patients, 20 were treated with photon CSI prior to 2015, and 19 were treated with proton CSI thereafter. Median age was 28 years (range 18–66). The molecular subtype was most commonly sonic hedgehog (68%). Patients most commonly received 36 Gy CSI in 20 fractions with a boost to 54–55.8 Gy (92%). Proton CSI delivered significantly lower mean doses to cochleae, lacrimal glands, lens, parotid glands, pharyngeal constrictors, esophagus, lungs, liver, and skin (all P &lt; .001). Patients receiving proton CSI had significantly lower rates of acute dysphagia of any grade (5% versus 35%, P = .044) and decreased median weight loss during radiation (+1.0 versus –2.8 kg, P = .011). Weight loss was associated with acute hospitalization (P = .009). Median follow-up was 2.9 and 12.9 years for proton and photon patients, respectively, limiting late toxicity and outcome comparisons. At the last follow-up, 5 photon patients had died (2 of progressive disease, 3 without recurrence ages 41–63) and 21% had experienced major cardiovascular events. At 10 years, 89% were alive and 82% were recurrence free. Conclusions This study demonstrates dosimetric improvements with proton CSI, potentially leading to decreased acute toxicity including dysphagia and weight loss during treatment.

No standard treatment paradigm exists for previously irradiated locally recurrent rectal cancer (PILRRC). Carbon-ion radiotherapy (CIRT) may improve oncologic outcomes and reduce toxicity compared with combined modality therapy (CMT). Eighty-five patients treated at Institution A with CIRT alone (70.4 Gy/16 fx) and eighty-six at Institution B with CMT (30 Gy/15 fx chemoradiation, resection, intraoperative electron radiotherapy (IOERT)) between 2006 and 2019 were retrospectively compared. Overall survival (OS), pelvic re-recurrence (PR), distant metastasis (DM), or any disease progression (DP) were analyzed with the Kaplan–Meier model, with outcomes compared using the Cox proportional hazards model. Acute and late toxicities were compared, as was the 2-year cost. The median time to follow-up or death was 6.5 years. Median OS in the CIRT and CMT cohorts were 4.5 and 2.6 years, respectively (p ≤ 0.01). No difference was seen in the cumulative incidence of PR (p = 0.17), DM (p = 0.39), or DP (p = 0.19). Lower acute grade ≥ 2 skin and GI/GU toxicity and lower late grade ≥ 2 GU toxicities were associated with CIRT. Higher 2-year cumulative costs were associated with CMT. Oncologic outcomes were similar for patients treated with CIRT or CMT, although patient morbidity and cost were lower with CIRT, and CIRT was associated with longer OS. Prospective comparative studies are needed.

Abstract Objective Patients with hormone receptor positive (HR+), HER2- metastatic breast cancer frequently require radiation therapy (RT) in addition to systemic therapy for disease control and symptom management. CDK 4/6 inhibitors are increasingly used in the care of hormone positive breast cancers. Stereotactic body radiotherapy (SBRT), a form of very focused high dose radiation, is also increasingly used in the care of metastatic breast cancer. Limited data have reported on the safety of palliative RT with CDK4/6 inhibitors, with some reports demonstrating synergistic effects with potential for increased toxicity. The objective of our study is to assess toxicity among patients who received SBRT, stereotactic radiosurgery (SRS), or other high dose RT along with CDK 4/6 inhibitors for treatment of metastatic breast cancer. Methods Women with metastatic breast cancer who received SBRT, SRS, or other higher dose RT with CDK 4/6 inhibitors at any point in their treatment between 2013 and 2021 were retrospectively identified. Timing of radiation therapy, either before, during, or after CDK 4/6 inhibitor use was assessed, as well as the interval between the two treatments. Treatment sites for radiotherapy included lung/chest, spine, and brain metastases. Physician assessed adverse events were obtained through clinical follow up and graded using CTCAE v5. Local control and OS were reported. Results Twenty patients met study inclusion criteria. Patients received radiation therapy and CDK 4/6 inhibitors concurrently (within 7 days) [n=7, 35%], within 30 days (RT pre-CDK 4/6 inhibitor use) [n=3, 15%], and greater than 30 days (RT pre-CDK 4/6 inhibitor use [n=6, 30%] and RT post-CDK 4/6 inhibitor use [n=4, 20%]). Radiation treatment sites were spine [n=10, 55%], brain [n=6, 30%], and lung/chest wall [n=3, 15%]. Radiation doses were as follows: 60 Gy in 5 fractions [n=1, 5%], 50 Gy in 5 fractions [n=2, 10%], 55 Gy in 20 fractions (chest wall recurrence) [n=1, 5%], 35 Gy in 15 fractions [n=1, 5%], 30 Gy in 10 fractions [n=6, 30%], 30 Gy in 3 fractions [n= 4, 20%], 24 Gy in 1 fraction [n=3, 15%], 20 Gy in 5 fractions [n=2, 10%]. Palbociclib was used in 85% of patients [n=17] and abemaciclib in 15% [n=3]. Median follow up for surviving patients was 4.7 years [IQR: 2.5-6.8 years]. Local control was achieved in 85% of patients [n=17]. Median OS was 2.7 years [IQR: 0.1-6.8 years]. Grade 2 toxicity [n=9, 45%] and grade 3 toxicity [n=3, 15%] was reported across all three treatment areas, with grade 2 or 3 fatigue [n=7, 35%] being the most frequently reported toxicity. In the spine treatment group, 1 patient who had CDK 4/6 inhibitor use within 30 days of radiotherapy experienced a grade 3 vertebral compression fracture as well as grade 3 fatigue. In the lung/chest wall treatment group, the patient treated to 55 Gy in 20 fractions to the chest wall with concurrent use had grade 3 skin/wound toxicity requiring prolonged wound care before healing. In the intra-cranial SRS group, grade 3 fatigue was reported in 1 patient with concurrent use. Conclusion The use of CDK 4/6 inhibitors concurrently with or within 30 days of radiation therapy was generally well tolerated with limited grade 3 toxicity. Two of three reports of grade 3 toxicity occurred with concurrent treatment. Due to the potential radiosensitizing properties of CDK 4/6 inhibitors, it may be prudent to hold combined treatment when clinically feasible. Further study is needed to validate these results. Citation Format: Kiran Chauhan, Roman O. Kowalchuk, Allison E. Garda, Dean A. Shumway, Mark R. Waddle, Robert Mutter, Kimberly Corbin. Safety of stereotactic body radiation therapy, stereotactic radiosurgery, and other radiation therapy with use of CDK 4/6 inhibitors for hormone receptor positive (HR+), HER2- metastatic breast cancer: a retrospective cohort study [abstract]. In: Proceedings of the 2022 San Antonio Breast Cancer Symposium; 2022 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2023;83(5 Suppl):Abstract nr P1-10-05.

AbstractBackgroundThis study was aimed at developing and validating a decision‐making tool predictive of overall survival (OS) for patients receiving stereotactic body radiation therapy (SBRT) for spinal metastases.MethodsThree hundred sixty‐one patients at one institution were used for the training set, and 182 at a second institution were used for external validation. Treatments most commonly involved one or three fractions of spine SBRT. Exclusion criteria included proton therapy and benign histologies.ResultsThe final model consisted of the following variables and scores: Spinal Instability Neoplastic Score (SINS) ≥ 6 (1), time from primary diagnosis < 21 months (1), Eastern Cooperative Oncology Group (ECOG) performance status = 1 (1) or ECOG performance status > 1 (2), and >1 organ system involved (1). Each variable was an independent predictor of OS (p < .001), and each 1‐point increase in the score was associated with a hazard ratio of 2.01 (95% confidence interval [CI], 1.79–2.25; p < .0001). The concordance value was 0.75 (95% CI, 0.71–0.78). The scores were discretized into three groups—favorable (score = 0–1), intermediate (score = 2), and poor survival (score = 3–5)—with 2‐year OS rates of 84% (95% CI, 79%–90%), 46% (95% CI, 36%–59%), and 21% (95% CI, 14%–32%), respectively (p < .0001 for each). In the external validation set (182 patients), the score was also predictive of OS (p < .0001). Increasing SINS<zaq;6> was predictive of decreased OS as a continuous variable (p < .0001).ConclusionsThis novel score is proposed as a decision‐making tool to help to optimize patient selection for spine SBRT. SINS may be an independent predictor of OS.

BackgroundPatients with high‐risk prostate cancer (HRPC) have multiple accepted treatment options. Because there is no overall survival benefit of one option over another, appropriate treatment must consider patient life expectancy, quality of life, and cost.MethodsThe authors compared quality‐adjusted life years (QALYs) and cost effectiveness among treatment options for HRPC using a Markov model with three treatment arms: (1) external‐beam radiotherapy (EBRT) delivered with 20 fractions, (2) EBRT with 23 fractions followed by low‐dose‐rate (LDR) brachytherapy boost, or (3) radical prostatectomy alone. An exploratory analysis considered a simultaneous integrated boost according to the FLAME trial (ClinicalTrials.gov identifier NCT01168479).ResultsTreatment strategies were compared using the incremental cost‐effectiveness ratio (ICER). EBRT with LDR brachytherapy boost was a cost‐effective strategy (ICER, $20,929 per QALY gained). These results were most sensitive to variations in the biochemical failure rate. However, the results still demonstrated cost effectiveness for the brachytherapy boost paradigm, regardless of any tested parameter ranges. Probabilistic sensitivity analysis demonstrated that EBRT with LDR brachytherapy was favored in 52% of 100,000 Monte Carlo iterations. In an exploratory analysis, EBRT with a simultaneous integrated boost was also a cost‐effective strategy, resulting in an ICER of $62,607 per QALY gained; however, it was not cost effective compared with EBRT plus LDR brachytherapy boost.ConclusionsEBRT with LDR brachytherapy boost may be a cost‐effective treatment strategy compared with EBRT alone and radical prostatectomy for HRPC, demonstrating high‐value care. The current analysis suggests that a reduction in biochemical failure alone can result in cost‐effective care, despite no change in overall survival.