Abdul‐Muhsin, Haidar

5108 Background: We previously proposed a hybrid framework combining LLMs and rule based algorithm (RBA) for automating risk stratification in locPCa. Herein, we aim to validate the risk stratification agent (RSA) in a prospective cohort, develop & evaluate the treatment recommendation agent (TRA) based on NCCN guidelines, and develop an interactive interface to facilitate clinicians for accurate risk stratification and treatment recommendations at the point of care. Methods: This study included pts with locPCa (2004-2024) presenting at Mayo Clinic with at least 1 positive prostate biopsy and MRI report available. For RSA prospective validation, GPT4 extracted key phenotypic variables (PSA, T stage, prostate volume, number of cores, Gleason patterns, grade group) from unstructured MRI and biopsy reports using a zeroshot prompt. An RBA then classified pts into NCCN risk groups. The agent performance was compared with the treating clinician's documentation and evaluated against gold-standard labels manually annotated by two independent clinicians. For development of TRA, two experiments were performed using GPT4 with and without retrieval-augmented generation (RAG) to generate treatment plans. Generated treatment plans were evaluated using NCCN guideline-informed treatment decision tree based algorithm (DTA). Evaluation metrics, weighted – accuracy (acc) and F1, were computed. A clinician facing interface (lisr.org/risk) was developed to provide accurate risk stratification and treatment plans. Results: A total of 858 pts were included (500 for prospective validation, 358 for treatment recommendations). Prospective validation for RSA demonstrated a higher F1 score of 0.89 compared to the treating clinician (F1: 0.58). Treatment recommendation experiments showed that GPT4 with RAG achieved higher acc (64% full – all correct treatment options and 36% partial – at least 1 correct treatment option) compared to GPT4 alone (35% and 65%, respectively). Sensitivity analysis using DTA-informed GPT4 note generation achieved 94% full acc and 6% partial acc. GPT4 alone generated hallucinated treatment options in 71% of cases, while GPT4 with RAG reduced this to 32%. Conclusions: The multi-agent framework based on LLM and RBA achieves high accuracy in risk stratification and treatment recommendation for locPCa. A multi-agent framework with an interactive interface holds high promise to enable efficient, accurate decision-making, and improve locPCa management at the point of care Performance evaluation. RSA(acc; F1) % TRA(full acc; partial acc) % Clinician RSA* GPT4 RAG+GPT4 DTA+GPT4 Overall 60; 58 89; 89 29; 71 64; 36 94; 6 V. High 94; 33 96; 96 55; 45 69; 31 93; 7 High 46; 52 92; 89 - - - Int. Unf 80; 61 91; 90 5; 95 38; 62 93; 7 Int. Fav 57; 64 84; 87 0; 100 98; 2 100; 0 Low 57; 65 87; 80 100; 0 95; 5 90; 10 V. Low 8; 13 50; 67^ - - - *Statistical significance (p<0.01) except for ^V. Low.

454 Background: Limited data exist on outcomes in mRCC patients with major venous involvement (MVI). Assessment of impact of a given variable on prognosis in cancer patients is challenging given competing causes of death (PMID: 25417239). This analysis aims to estimate cancer prognosis in mRCC patients with MVI, considering death from competing causes. Methods: Data from the Surveillance Epidemiology and End Results (SEER) database (2010-2020) were analyzed for survival and mortality in mRCC patients with MVI. 5-year (y) overall survival (OS) and 5-y probability of cancer death were calculated using the actuarial method in SEER*Stat 8.4.4, reported as percentages with 95% confidence intervals (CI). Results were based on the site of MVI (renal vein, inferior vena cava [IVC]) and stratified by age (<65, >65 years), gender (male, female), and race/ethnicity (Non-Hispanic [NH] White, NH Black, NH Asian/Pacific Islander [API], Hispanic). Results: A total of 89047 mRCC patients were included in our analysis of whom 10785 (12%) had major venous involvement (MVI). Overall, patients who experienced MVI had worse prognosis as compared to those who did not have MVI (5-y OS: 53.2% vs 84.4% and 5-y cancer death: 38.6% vs 8.2% respectively). Among cohort with MVI, prognosis was poor with IVC involvement as compared to involvement of renal vein (5-y OS: 36% vs 57%; 5-y cancer death: 56% vs 35% respectively). Patients who had the worst prognosis with MVI included older adults (>65 y) (5-y OS: 47.8%; 5-y cancer death: 40.8%) and NH Black (5-y OS: 41.5%; 5-y cancer death: 49.8%) (Table). Conclusions: Prognosis in mRCC with MVI, especially with IVC involvement, was poor, particularly for older adults (>65 y) and NH Black patients. Further research is needed for better prognostication and risk stratification to improve outcomes for high-risk patients. Subgroup No MVI MVI N 5-year OS (%), 95% CI 5-year Cancer death (%), 95% CI N 5-year OS (%), 95% CI 5-year Cancer death (%), 95% CI Overall population 78,262 84.4 (84.1-84.7) 8.2 (8.0-8.4) 10,785 53.2 (52.0-54.3) 38.6 (37.5-39.7) Age <65 y 49,217 89.0 (88.6-89.2) 6.5 (6.3-6.8) 5,897 57.5 (56.0-59.0) 36.8 (35.4-38.3) >65 y 29,045 77.0 (76.1-77.2) 10.9 (10.5-11.4) 4,888 47.8 (46.0-49.5) 40.8 (39.2-42.4) Gender Male 49,221 83.0 (82.7-83.5) 8.9 (8.6-9.2) 7,503 53.2 (51.8-54.6) 38.3 (37.0-39.6) Female 29,041 86.5 (86.0-86.9) 7.0 (6.7-7.4) 3,282 53.0 (51.0-55.0) 39.4 (37.4-41.3) Race/ethnicity NH White 49,520 84.0 (83.7-84.4) 8.3 (8.0-8.5) 7,182 53.5 (52.1-54.8) 38.0 (36.7-39.3) NH Black 8,878 82.0 (81.0-82.9) 8.1 (7.5-8.8) 685 41.5 (37.1-45.9) 49.8 (45.4-54.1) NH API 4,747 86.0 (85.1-87.4) 8.5 (7.6-9.4) 713 52.9 (48.2-57.3) 39.9 (35.5-44.1) Hispanic (All Races) 13,826 86.5 (85.8-87.2) 7.9 (7.4-8.4) 2,014 56.2 (53.4-58.8) 36.7 (34.1-39.2) N: number of cases; OS: overall survival; CI: confidence interval; NH: Non-Hispanic; API: Asian/Pacific Islander; MVI: major venous involvement.

136 Background: The ECOG performance status (PS) is a critical measure for evaluating functional capacity and determining clinical trial eligibility in patients with advanced prostate cancer. We aim to assess the capability of large language models (LLMs) to accurately infer ECOG PS from unstructured oncology notes. Methods: This retrospective study included patients with metastatic castration-resistant prostate cancer (mCRPC) receiving Lutetium-177–PSMA-617 ( 177 Lu) at the Mayo Clinic (2022-2023). Baseline ECOG PS scores were manually curated by a trained clinician from clinical notes recorded within four weeks prior to 177 Lu initiation. Structured zero-shot prompts were then used to prompt GPT-4 to infer ECOG PS scores from unstructured oncology notes. Total dataset was randomly sampled to prompt development (20%) and held-out test set (80%). Three experimental setups were employed to assess model performance: (i) utilizing unmasked clinical notes, (ii) clinical notes with only numerical ECOG PS scores masked while retaining narrative descriptive/assessment by oncologist of ECOG score, and (iii) clinical notes with both numerical scores and narrative ECOG descriptors masked. Final performance was assessed using evaluation metrics (precision, recall, F1 score) on the held-out test set. Results: A total of 240 patients (n: 50 prompt development; n: 190 test set) were included in the analysis. The median age was 70 (IQR: 66-76); a majority of patients were White (n: 229; 95%) and non-Hispanic (n: 233; 97%). The most prevalent ECOG PS score was 0 (n: 140; 58%), followed by 1 (n: 88; 37%), and 2 (n: 12; 5%). The performance was numerically higher when using unmasked notes (weighted average precision: 0.89; recall: 0.89; F1: 0.89), compared to using notes with masked scores only (weighted average precision: 0.76; recall: 0.79; F1: 0.75) and notes with both scores and narrative masked (weighted average precision: 0.57; recall: 0.53; F1: 0.48). Metrics by each ECOG PS score is outlined (Table). Conclusions: Large language models demonstrate significant potential in precisely extracting ECOG performance status from unstructured oncology notes, offering valuable support for treatment decision-making and clinical trial enrollment. However, model performance declines as reliance on implicit reporting within clinical notes increases. Future efforts focusing on prediction of ECOG using a series of notes over time will be the next step. ECOG Unmasked oncology notes ECOG Score masked only ECOG Score + ECOG narrative marked Precision Recall F1 Score Precision Recall F1 Score Precision Recall F1 Score 0 0.89 0.92 0.91 0.75 0.93 0.83 0.59 0.74 0.66 1 0.88 0.86 0.87 0.76 0.74 0.75 0.19 0.47 0.27 2 1.00 0.82 0.90 0.78 0.33 0.47 0.78 0.16 0.26

132 Background: Distant metastasis at the time of prostate cancer (PCa) diagnosis is associated with increased morbidity and mortality. This study aimed to assess the impact of sociodemographic characteristics on distant metastasis at the time of PCa diagnosis. Methods: The Surveillance Epidemiology and End Results (SEER) database (2000-2021), including PCa patients from 17 registries, was queried. Patients with late-stage diagnosis (LSD) were defined as those who were initially diagnosed with stage IV disease, indicating distant metastasis. Sociodemographic predictors included race, ethnicity, age, marital status, residential status, and median household income. Multicollinearity was assessed using variance inflation factors (VIF). Univariable logistic regression was performed to identify significant predictors of LSD, which were further analyzed in multivariable logistic regression using a maximum likelihood estimation approach. Odds ratios with 95% confidence intervals (CI) were computed with a p-value of <0.05 indicating a statistically significant association. Results: Of the 1,089,736 PCa patients diagnosed, 64,983 (6%) had an LSD. The majority were white (80%), non-Hispanic/Latino (HL) (91%), aged ≥65 years (60%), married (75%), residents in metropolitan areas (88%), and had an annual house income of ≥$70,000 (70%). Multivariable logistic regression identified Black (OR: 1.24; 95%CI: 1.22-1.27), Asian or Pacific Islander (1.35; 1.30-1.40), and American Indian/Alaska Native (1.70; 1.53-1.89) to have higher odds of LSD compared to White patients (Table). In addition, HL ethnicity compared to non-HL (OR: 1.45; 95% CI: 1.42-1.49), age ≥65 compared to <65 (1.68; 1.65-1.71), unmarried status compared to married (1.88; 1.85-1.91), residency in non-metropolitan areas compared to metropolitan (1.10; 1.07-1.12), and household income <$70,000 compared to ≥$70,000 (1.03; 1.01-1.05) had a significant association with LSD. Conclusions: Older, non-White (Black, Asian/Pacific Islander, American Indian/Alaska Native), Hispanic/Latino, unmarried men, residents of non-metropolitan areas, or those with lower annual household incomes have higher odds of late-stage diagnosis. Interventions targeting early detection in these at-risk groups may improve prostate cancer outcomes in the US. Variable Odds Ratio (95% CI) p-value Race (Black vs White) 1.24 (1.22-1.27) <0.01 Race (Asian or Pacific Islander vs White) 1.35 (1.31-1.40) <0.01 Race (American Indian/Alaska Native vs White) 1.70 (1.53-1.89) <0.01 Ethnicity (Hispanic/Latino vs non-/Hispanic/Latino) 1.45 (1.42-1.49) <0.01 Age (≥65 vs <65) 1.68 (1.65-1.71) <0.01 Marital status (unmarried vs married) 1.88 (1.85-1.91) <0.01 Residence status (non-metropolitan vs metropolitan) 1.10 (1.07-1.12) <0.01 Household income (<$70,000 vs ≥$70,000) 1.03 (1.01-1.05) <0.01

341 Background: Specialist clinicians (oncologists, urologists, radiation oncologists) read through free text reports of prostate MRI and biopsy reports to risk stratify patients during initial consultation of localized PCa. This laborious and time-consuming process is repeated for each day during the clinic. Herein, we propose a hybrid stratification framework utilizing a combination of large language model (LLM) and rule-based programming to automate this process. Methods: This retrospective study included patients with localized PCa (2004-2024) presenting to care at Mayo Clinic with at least one positive prostate biopsy and MRI report available. The state-of-the-art LLM – GPT4 – was utilized using a structured zeroshot prompt to extract relevant phenotypic variables (clinical T stage, Gleason patterns, grade group, extent of specimen/cores involved, prostate size/volume) from unstructured MRI and biopsy reports. Using a rule-based algorithm, the extracted variables and PSA at PCa diagnosis were used to categorize each patient into one of the six NCCN risk categories (very low [VL], low [L], favorable intermediate [FInt], unfavorable intermediate [UFInt], high risk (HR), very high risk [VHR]). Prompts were iteratively developed using ~5% of the total dataset and validated on ~10% of the dataset. Final performance was assessed against a held-out expert annotated test dataset using evaluation metrics (accuracy, F1-score, precision, recall). Additionally, manual annotation was conducted by two independent novice reviewers to compare machine annotated and novice-human annotated risk categorization. Results: A total of 397 patients were included in the evaluation. The median age at diagnosis was 64.8 (IQR: 59.7-68.5); majority of the men were White (n: 366; 92%) and non-Hispanic (n: 371; 94%). The most prevalent risk category was UFint (n: 151; 38%) followed by FInt (n: 80; 20.2%), HR (n: 76; 19.1%), VHR (n: 64; 16.1%), L (21: 5.3%), and VL (n: 5; 1.3%). Novice reviewers achieved an accuracy of 79%, precision of 80%, recall of 82% and a F1-score of 0.81. The hybrid stratification framework using GPT4 achieved an accuracy of 89%, precision of 89%, recall of 88% and a F1-score of 0.88. Among 43 errors by GPT4 risk stratification, majority of the errors were due to miscategorization of clinical T-stage (n: 21; 49%) followed by erroneous number of positive cores (n: 13; 30%). However, additional evaluation showed that GPT4 achieved numerically higher performance (accuracy: 90%; precision: 91%; recall: 90%; F1: 0.90) compared to novice reviewers (accuracy: 77%; precision: 89%; recall: 77%; F1: 0.82) for ascertaining clinical T-stage. Conclusions: Large language models exhibited superior performance than novice clinicians for risk stratification in patients with localized PCa. As the next step, we intend to validate our framework for automated risk classification in a prospective manner.

428 Background: Prostate cancer management presents a significant healthcare burden, with the need to efficiently triage patients for treatment. Our objective is to leverage large language models to predict physician-recommended treatment plans from unstructured clinical notes. By accurately predicting treatment plans, we aim to risk stratify and triage patients effectively, thereby optimizing the allocation of physician resources. Methods: 448 unstructured initial urology consultation patient notes following first positive prostate cancer biopsy were identified. The recommended and final treatments received were manually annotated to establish ground truth labels (Table 1). The dataset was split 80:20 for training and testing, preprocessed to remove plan sections and formatted into question-answer (QA) format. A domain-specific large language model (LLM) inspired by GPT and a specialized tokenizer (PCa- LLM) for prostate cancer terminology were developed. QA models were built using the PCa-LLM and compared with those using GPT-2 as the backbone to predict recommended and final treatments. Results: For the physician-recommended treatment plans, our LLM (PCa-LLM) showed superior performance with higher AUROC scores for curative vs. non-curative treatments (0.78 vs. 0.65), chemo-hormonal vs. other non-curative treatments (0.89 vs. 0.65), and surveillance vs. all other treatments (0.72 vs. 0.70), while both models achieved the same high AUROC of 0.99 for chemo-hormonal vs. all other treatments. For final treatments, PCa-LLM demonstrated better AUROC for curative vs. non-curative treatments (0.77 vs. 0.74) and chemo-hormonal vs. other non-curative treatments (0.71 vs. 0.66), while GPT2 outperformed PCa-LLM for surveillance vs. all other treatments (0.78 vs. 0.70). Both models achieved an AUROC of 0.99 for chemo-hormonal vs. all other treatments. Conclusions: PCa-LLM accurately predicted most treatment categories better than GPT2, with higher AUROC scores, and can be utilized to triage prostate cancer patients using initial consultation notes. Task Physician -Recommended Treatment Plan Final Treatment Received Curative Prostatectomy/Radiation 228 230 Non-Curative Focal Therapy 22 15 Active Surveillance 40 45 Chemo-hormonal 30 30 Model Predictions (AUROC) GPT2 PCa-LLM GPT2 PCa-LLM Curative vs. non-curative 0.65 0.78 0.74 0.77 Chemohormonal vs. other non-curative 0.65 0.89 0.66 0.71 Chemohormonal vs. all other 0.99 0.99 0.99 0.99 Surveillance vs. other non-curative 0.64 0.60 0.67 0.59 Surveillance vs. all other 0.70 0.72 0.78 0.70

Abstract The study of prehabilitation and rehabilitation ([p]rehabilitation) to alleviate the sequelae of bladder cancer and its treatment has generated numerous opportunities to improve the quality of life of bladder cancer survivors. The authors conducted a scoping review of randomized clinical trials (RCTs) to identify knowledge gaps in and research directions for (p)rehabilitative support for those affected by bladder cancer. The authors systematically searched six databases and synthesized key findings from RCTs conducted from January 1, 2004, through March 15, 2022, that enrolled participants with bladder cancer, survivors, or caregivers in outpatient (p)rehabilitative programs (e.g., exercise, nutrition, or psychological support). Outcomes were characterized according to eight prespecified, clinically relevant categories. The search retrieved 10,968 records, 27 of which met the inclusion criteria, and 24 described unique RCTs with 2471 enrolled participants. Of 24 interventions, 17 (71%) yielded statistically significant results for the outcome of interest. Only one RCT included a cost‐effectiveness analysis, and only two characterized the efficacy of interventions for caregivers. Of 11 RCTs involving psychological support, eight yielded statistically significant results, as did nine of 11 RCTs with physical exercise interventions, three of four RCTs with educational interventions, three of four RCTs with nutritional support interventions, one of two RCTs with pharmacologic medications, and zero of one RCT with physical therapy. The most promising interventions for inclusion in multimodal, personalized (p)rehabilitation programs included exercise, stress management training, cognitive training, smoking and alcohol cessation counseling, immunonutrition, stoma education, and penile rehabilitation. Further studies of the cost effectiveness and efficacy for caregivers of such interventions are needed. Plain Language Summary In a scoping review of all randomized clinical trials involving prehabilitative or rehabilitative diet, exercise, and psychological support interventions for patients with bladder cancer, survivors, and their caregivers, 17 of 24 (71%) interventions yielded statistically significant improvements in the outcome of interest. Clinicians should consider implementing such interventions for those affected by bladder cancer.

Background: Differential kidney function assessment is an important part of preoperative evaluation of various urological interventions. It is obtained through dedicated nuclear medical imaging and is not yet implemented through conventional Imaging. Objective: We assess if differential kidney function can be obtained through evaluation of contrast-enhanced computed tomography(CT) using a combination of deep learning and (2D and 3D) radiomic features. Methods: All patients who underwent kidney nuclear scanning at Mayo Clinic sites between 2018–2022 were collected. CT scans of the kidneys were obtained within a 3-month interval before or after the nuclear scans were extracted. Patients who underwent a urological or radiological intervention within this time frame were excluded. A segmentation model was used to segment both kidneys. 2D and 3D radiomics features were extracted and compared between the two kidneys to compute delta radiomics and assess its ability to predict differential kidney function. Performance was reported using receiver operating characteristics, sensitivity, and specificity. Results: Studies from Arizona & Rochester formed our internal dataset ( n = 1,159). Studies from Florida were separately processed as an external test set to validate generalizability. We obtained 323 studies from our internal sites and 39 studies from external sites. The best results were obtained by a random forest model trained on 3D delta radiomics features. This model achieved an area under curve (AUC) of 0.85 and 0.81 on internal and external test sets, while specificity and sensitivity were 0.84,0.68 on the internal set, 0.70, and 0.65 on the external set. Conclusion: This proposed automated pipeline can derive important differential kidney function information from contrast-enhanced CT and reduce the need for dedicated nuclear scans for early-stage differential kidney functional assessment. Clinical Impact: We establish a machine learning methodology for assessing differential kidney function from routine CT without the need for expensive and radioactive nuclear medicine scans.

98 Background: The risk of secondary primary malignancies (SPM) among the survivors of GU cancer is not well-established. In this study, we attempt to describe the patterns of SPMs in patients diagnosed with GU cancers. Methods: The Surveillance, Epidemiology, and End Results (SEER) database (SEER*Stat 8.4.2) was searched to obtain data on the incidence of SPM in adult patients (age >18 years) diagnosed with GU cancers, including prostate, renal, and bladder cancer. Standardized incidence ratios (SIRs) with a 95% confidence interval (CI) were calculated. SIR was defined as the observed SPMS from each category divided by the expected number of SPMs in the age-matched US population for the same period. The data was further stratified by age (<50y, 50-70y, >70y), race (White, Black, Asian/Pacific Islander), ethnicity (Hispanic, non-Hispanic), chemotherapy, and radiotherapy exposure. Results: A total of 1,041,217 patients were included in this analysis. In patients diagnosed with GU cancer, the incidence of overall SPM was 11%. Particularly, patients with GU cancers were observed to have a high incidence of Endocrine system SPM (1.52; 1.45-1.59), and urinary system SPM (1.51; 1.49-1.53) These results were consistent by different sociodemographic groups. In patients diagnosed with prostate cancer, the incidence of thyroid SPM was increased (1.31; 1.23-1.39). Particularly, prostate cancer patients treated with radiation therapy had increased incidence rates of SPM in the peritoneum, omentum and mesentery (1.57; 0.79-2.81) and urinary bladder (1.36;1.32-1.40) was higher as compared to those with no exposure to radiation therapy (1.11;1.08-1.13 and 1.03;1-1.06 respectively). Bladder cancer patients had an overall increased incidence of SPM in the esophagus (1.21; 1.08-1.35), lung, bronchus, trachea, and other respiratory organs (1.97;1.92-2.01), prostate (3.50;3.4-3.6) and kidney (1.86;1.76-1.97). A higher incidence of renal pelvis, ureter and other urinary organ SPM (11.08;10.48-11.07) was also notable in this population. Similarly, kidney cancer patients had a higher incidence of the endocrine system (3.00;2.74-3.27), prostate (1.24;1.2-1.29) and bladder 1.78;1.68-1.79 SPMs. Conclusions: This population-based study shows an increased risk of SPM development in GU cancer survivors compared to the general population. Clinicians should keep a high index of suspicion for SPMs and initiate appropriate workups at early warning signs.