Riaz, Irbaz Bin

136 Background: The ECOG performance status (PS) is a critical measure for evaluating functional capacity and determining clinical trial eligibility in patients with advanced prostate cancer. We aim to assess the capability of large language models (LLMs) to accurately infer ECOG PS from unstructured oncology notes. Methods: This retrospective study included patients with metastatic castration-resistant prostate cancer (mCRPC) receiving Lutetium-177–PSMA-617 ( 177 Lu) at the Mayo Clinic (2022-2023). Baseline ECOG PS scores were manually curated by a trained clinician from clinical notes recorded within four weeks prior to 177 Lu initiation. Structured zero-shot prompts were then used to prompt GPT-4 to infer ECOG PS scores from unstructured oncology notes. Total dataset was randomly sampled to prompt development (20%) and held-out test set (80%). Three experimental setups were employed to assess model performance: (i) utilizing unmasked clinical notes, (ii) clinical notes with only numerical ECOG PS scores masked while retaining narrative descriptive/assessment by oncologist of ECOG score, and (iii) clinical notes with both numerical scores and narrative ECOG descriptors masked. Final performance was assessed using evaluation metrics (precision, recall, F1 score) on the held-out test set. Results: A total of 240 patients (n: 50 prompt development; n: 190 test set) were included in the analysis. The median age was 70 (IQR: 66-76); a majority of patients were White (n: 229; 95%) and non-Hispanic (n: 233; 97%). The most prevalent ECOG PS score was 0 (n: 140; 58%), followed by 1 (n: 88; 37%), and 2 (n: 12; 5%). The performance was numerically higher when using unmasked notes (weighted average precision: 0.89; recall: 0.89; F1: 0.89), compared to using notes with masked scores only (weighted average precision: 0.76; recall: 0.79; F1: 0.75) and notes with both scores and narrative masked (weighted average precision: 0.57; recall: 0.53; F1: 0.48). Metrics by each ECOG PS score is outlined (Table). Conclusions: Large language models demonstrate significant potential in precisely extracting ECOG performance status from unstructured oncology notes, offering valuable support for treatment decision-making and clinical trial enrollment. However, model performance declines as reliance on implicit reporting within clinical notes increases. Future efforts focusing on prediction of ECOG using a series of notes over time will be the next step. ECOG Unmasked oncology notes ECOG Score masked only ECOG Score + ECOG narrative marked Precision Recall F1 Score Precision Recall F1 Score Precision Recall F1 Score 0 0.89 0.92 0.91 0.75 0.93 0.83 0.59 0.74 0.66 1 0.88 0.86 0.87 0.76 0.74 0.75 0.19 0.47 0.27 2 1.00 0.82 0.90 0.78 0.33 0.47 0.78 0.16 0.26

134 Background: Prostate cancer patients with visceral metastases at diagnosis are often treated using one-size-fits-all approach. For example, CHAARTED criteria treats any visceral metastasis as high volume. Therefore, we aimed to assess overall survival (OS) by site of visceral involvement in patients with metastatic PCa. Methods: Surveillance, Epidemiology, and End Results (SEER) database (2010-2021) was queried to obtain case listing data on metastatic PCa patients from 17 registries. The patients were categorized into three groups based on visceral metastasis at diagnosis: lung metastasis, liver metastasis, and others. Kaplan-Meier survival analysis was performed to estimate median OS with 95% confidence interval (CI). Log-rank test was conducted to assess if survival was significantly different among the groups. Cox proportional hazard regression analysis was performed to assess the magnitude of difference among the groups. P-value <0.05 established statistical significance. Results: This analysis included 21,007 metastatic PCa patients (lung: n=319; liver: n=123; brain: 107; bone: 20538). Metastatic PCa population with lung metastasis exhibited a median OS of 60 months (95% CI: 48-72) compared to 24 months (95% CI: 12-36) in population with liver metastases and to 36 months (95% CI: 36-36) in population with other metastases. The difference in OS among the groups was statistically significant (p < 0.001). Compared to population with lung metastasis, those with liver (HR: 1.91; 95% CI: 1.41-2.57) but not those with other metastases (1.16; 0.97-1.39) had significantly worse OS adjusting for Gleason score, PSA and race (Table). Conclusions: In metastatic PCa, patients with lung-only metastasis have significantly better overall survival and should be considered differently in prognostication to liver metastases. Consideration of site of visceral metastasis is critical for treatment intensification decisions. Limitations of this analysis include lack of accounting for confounding relationships at individual patient level. Variable Hazard Ratio (95% CI) Metastases (Lung) REFERENCE Metastases (Liver) 1.91 (1.41-2.57) Metastases (Other) 1.16 (0.97-1.39) Gleason score≥8 REFERENCE Gleason score <8 0.68 (0.64 -0.72) PSA - ng/dL 1.01 (1.00 – 1.01) Race (White) REFERENCE Race (Black) 0.99 (0.94 – 1.05) Race (Asian or Pacific Islander) 0.70 (0.64 – 0.76)

132 Background: Distant metastasis at the time of prostate cancer (PCa) diagnosis is associated with increased morbidity and mortality. This study aimed to assess the impact of sociodemographic characteristics on distant metastasis at the time of PCa diagnosis. Methods: The Surveillance Epidemiology and End Results (SEER) database (2000-2021), including PCa patients from 17 registries, was queried. Patients with late-stage diagnosis (LSD) were defined as those who were initially diagnosed with stage IV disease, indicating distant metastasis. Sociodemographic predictors included race, ethnicity, age, marital status, residential status, and median household income. Multicollinearity was assessed using variance inflation factors (VIF). Univariable logistic regression was performed to identify significant predictors of LSD, which were further analyzed in multivariable logistic regression using a maximum likelihood estimation approach. Odds ratios with 95% confidence intervals (CI) were computed with a p-value of <0.05 indicating a statistically significant association. Results: Of the 1,089,736 PCa patients diagnosed, 64,983 (6%) had an LSD. The majority were white (80%), non-Hispanic/Latino (HL) (91%), aged ≥65 years (60%), married (75%), residents in metropolitan areas (88%), and had an annual house income of ≥$70,000 (70%). Multivariable logistic regression identified Black (OR: 1.24; 95%CI: 1.22-1.27), Asian or Pacific Islander (1.35; 1.30-1.40), and American Indian/Alaska Native (1.70; 1.53-1.89) to have higher odds of LSD compared to White patients (Table). In addition, HL ethnicity compared to non-HL (OR: 1.45; 95% CI: 1.42-1.49), age ≥65 compared to <65 (1.68; 1.65-1.71), unmarried status compared to married (1.88; 1.85-1.91), residency in non-metropolitan areas compared to metropolitan (1.10; 1.07-1.12), and household income <$70,000 compared to ≥$70,000 (1.03; 1.01-1.05) had a significant association with LSD. Conclusions: Older, non-White (Black, Asian/Pacific Islander, American Indian/Alaska Native), Hispanic/Latino, unmarried men, residents of non-metropolitan areas, or those with lower annual household incomes have higher odds of late-stage diagnosis. Interventions targeting early detection in these at-risk groups may improve prostate cancer outcomes in the US. Variable Odds Ratio (95% CI) p-value Race (Black vs White) 1.24 (1.22-1.27) <0.01 Race (Asian or Pacific Islander vs White) 1.35 (1.31-1.40) <0.01 Race (American Indian/Alaska Native vs White) 1.70 (1.53-1.89) <0.01 Ethnicity (Hispanic/Latino vs non-/Hispanic/Latino) 1.45 (1.42-1.49) <0.01 Age (≥65 vs <65) 1.68 (1.65-1.71) <0.01 Marital status (unmarried vs married) 1.88 (1.85-1.91) <0.01 Residence status (non-metropolitan vs metropolitan) 1.10 (1.07-1.12) <0.01 Household income (<$70,000 vs ≥$70,000) 1.03 (1.01-1.05) <0.01

341 Background: Specialist clinicians (oncologists, urologists, radiation oncologists) read through free text reports of prostate MRI and biopsy reports to risk stratify patients during initial consultation of localized PCa. This laborious and time-consuming process is repeated for each day during the clinic. Herein, we propose a hybrid stratification framework utilizing a combination of large language model (LLM) and rule-based programming to automate this process. Methods: This retrospective study included patients with localized PCa (2004-2024) presenting to care at Mayo Clinic with at least one positive prostate biopsy and MRI report available. The state-of-the-art LLM – GPT4 – was utilized using a structured zeroshot prompt to extract relevant phenotypic variables (clinical T stage, Gleason patterns, grade group, extent of specimen/cores involved, prostate size/volume) from unstructured MRI and biopsy reports. Using a rule-based algorithm, the extracted variables and PSA at PCa diagnosis were used to categorize each patient into one of the six NCCN risk categories (very low [VL], low [L], favorable intermediate [FInt], unfavorable intermediate [UFInt], high risk (HR), very high risk [VHR]). Prompts were iteratively developed using ~5% of the total dataset and validated on ~10% of the dataset. Final performance was assessed against a held-out expert annotated test dataset using evaluation metrics (accuracy, F1-score, precision, recall). Additionally, manual annotation was conducted by two independent novice reviewers to compare machine annotated and novice-human annotated risk categorization. Results: A total of 397 patients were included in the evaluation. The median age at diagnosis was 64.8 (IQR: 59.7-68.5); majority of the men were White (n: 366; 92%) and non-Hispanic (n: 371; 94%). The most prevalent risk category was UFint (n: 151; 38%) followed by FInt (n: 80; 20.2%), HR (n: 76; 19.1%), VHR (n: 64; 16.1%), L (21: 5.3%), and VL (n: 5; 1.3%). Novice reviewers achieved an accuracy of 79%, precision of 80%, recall of 82% and a F1-score of 0.81. The hybrid stratification framework using GPT4 achieved an accuracy of 89%, precision of 89%, recall of 88% and a F1-score of 0.88. Among 43 errors by GPT4 risk stratification, majority of the errors were due to miscategorization of clinical T-stage (n: 21; 49%) followed by erroneous number of positive cores (n: 13; 30%). However, additional evaluation showed that GPT4 achieved numerically higher performance (accuracy: 90%; precision: 91%; recall: 90%; F1: 0.90) compared to novice reviewers (accuracy: 77%; precision: 89%; recall: 77%; F1: 0.82) for ascertaining clinical T-stage. Conclusions: Large language models exhibited superior performance than novice clinicians for risk stratification in patients with localized PCa. As the next step, we intend to validate our framework for automated risk classification in a prospective manner.

108 Background: PARPi are approved for the treatment of pts with mCRPC either as single agents or as a combination with an androgen receptor pathway inhibitor (ARPI) [PMID: 37442702]. In a US Food and Drug Administration (FDA) pooled analysis, the benefit from PARPi appeared greatest for pts harboring BRCA1/2 alterations [PMID: 38484203]. However, real-world uptake of PARPi in pts with mCRPC with BRCA1/2 alterations is unknown. Herein, our objective was to assess the usage of PARPi in real-world pts with mCRPC harboring BRCA1/2 alterations. Methods: A de-identified nationwide Flatiron Health electronic health record (EHR)-derived database was used to extract pt-level data. Eligibility criteria: pts with mCRPC harboring alterations in BRCA1 or BRCA2 or both and alive after 8/15/2020 (i.e., 3 months following the approval of the first PARPi, rucaparib, in mCRPC) with available treatment information. The data cutoff date was 5/31/2024. Pts were categorized into two cohorts based on the receipt of any PARPi or not. Baseline characteristics at the time of mCRPC diagnosis, including age, race-ethnicity, and insurance plan, were collected in both cohorts. Results: Of the overall cohort of 24,105 pts with metastatic prostate cancer, 443 pts had mCRPC with BRCA1/2 alterations and were eligible and included in this analysis. 227 (51.2%) received a PARPi, while 216 did not (48.8%). Among the 227 pts who received a PARPi, 166 received it as a single agent (73.1%), 40 received it in combination with an ARPI (17.6%), and 21 received it in combination with other agents (9.3%). The median age was 72 in both cohorts (IQR 65 – 79 in patients receiving a PARPi and 66 – 78 in patients who did not). Other baseline characteristics in both groups are shown (Table). Conclusions: Despite PARPi demonstrating survival improvement in pts with mCRPC with BRCA1/2 alterations, a significant proportion of pts do not receive them. Our findings highlight the need to improve education among clinicians of level 1 evidence and improve access to life-prolonging therapies in pts with mCRPC. Baseline characteristics of pts with mCRPC who received vs. did not receive a PARPi. Characteristic Subgroup Pts who received a PARPiN = 227 Pts who did not receive a PARPiN = 216 Race-ethnicity, n (%) Asian 9 (4) 1 (0.5) Black 22 (9.7) 30 (13.9) Hispanic-Latino 10 (4.4) 10 (4.6) White 149 (65.6) 124 (57.4) Other and unknown 37 (16.3) 51 (23.6) Insurance, n (%) Commercial health plan 192 (84.5) 182 (84.3) Medicare-other government programs 31 (13.7) 18 (8.3) Medicaid 1 (0.4) 3 (1.4) Others and unknown 3 (1.3) 13 (6)

184 Background: We update evidence on mHSPC when new evidence becomes available (PMID: 36862387). ARANOTE trial prompted us to assess the comparative efficacy of darolutamide (DARO) with other ARPi agents in mHSPC using the living network meta-analysis (LNMA). Methods: This LNMA is conducted using the living interactive evidence (LIvE) synthesis framework. Phase III trials assessing systemic treatments in mHSPC are included. Mixed treatment comparisons were made using a frequentist network meta-analysis. P-scores were computed to assess relative treatment rankings with higher values indicating potentially better efficacy. Results: This report of LNMA includes a total 11 trials (12628 patients; 12 unique treatment options) as of October 1 st , 2024. Overall results were consistent with prior updates. Analysis limited to the ARPi trials showed that inhigh volume, abiraterone acetate (AAP)+androgen deprivation therapy (ADT) (HR: 0.46; 95% CI: 0.40-0.53; rank 1), apalutamide (APA)+ADT (0.53; 0.41-0.68; rank 3), DARO+ADT (0.60; 0.44-0.81; rank 4) and enzalutamide (E)+ADT (0.48; 0.41-0.56; rank 2) significantly improved radiographic progression-free survival (rPFS) compared to ADT alone. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (1.30; 0.94-1.82), APA+ADT (1.14; 0.77-1.67) and E+ADT (1.24; 0.89-1.79). Inlow volume, AAP+ADT (0.48; 0.37-0.63; rank 4), APA+ADT (0.36; 0.22-0.58; rank 3), DARO+ADT (0.30; 0.15-0.60; rank 1) and E+ADT (0.32; 0.25-0.41; rank 2) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.67; 0.47-0.96) in low volume. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (0.62; 0.30-1.30), APA+ADT (0.83; 0.36-1.92) and E+ADT (0.93; 0.45-1.94). In synchronous disease, AAP+ADT (0.58; 0.51-0.67; rank 4), APA+ADT (0.49; 0.39-0.62; rank 2), DARO+ADT (0.59; 0.44-0.79; rank 3) and E+ADT (0.42; 0.36-0.50; rank 1) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.73; 0.59-0.89) in synchronous disease. In metachronous disease, DARO+ADT (0.34; 0.17-0.67; rank 1), APA+ADT (0.41; 0.22-0.77; rank 2) and E+ADT (0.44; 0.35-0.57; rank 3) significantly improved rPFS compared to ADT. No statistically significant differences were observed with DARO+ADT compared to APA+ADT (0.83; 0.33-2.08) and E+ADT (0.76; 0.37-1.57) in metachronous disease. The results were consistent for overall survival. Conclusions: Current evidence suggests no significant differences for darolutamide as compared to other ARPi agents. Enzalutamide may be preferred over AAP in low-volume or synchronous disease. Given similar efficacy, choice of ARPi requires careful consideration of patient-specific factors including cost, accessibility and toxicity.

75 Background: The Rx landscape of pts with mCRPC has recently evolved with the approval of lutetium-177-PSMA-617 (Lu-177) and poly(ADP) ribose polymerase inhibitors (PARPi) either as single agents or as combinations with an androgen receptor pathway inhibitor (ARPI) [PMID: 37442702]. However, real-world data on the uptake of these agents are lacking. Herein, we sought to assess the Rx patterns and attrition rates of pts with mCRPC in the era of newly approved therapies. Methods: The de-identified nationwide Flatiron Health Electronic-Health Record (EHR)-derived database was used to extract pt-level data. Eligibility criteria: diagnosis of mCRPC and Rx line (L) information. The data cut-off date was 5/31/2024. Rx patterns in each line of therapies were summarized using frequency and percentages. All analysis was done using R version 4.2.3. Results: Of the overall cohort of 24,105 pts with metastatic prostate cancer, 12,333 pts diagnosed with mCRPC between 1/1/2013 and 5/9/2024 met the eligibility criteria and were included. Rx trends are summarized (Table). Of pts receiving 1L Rx, 61% of pts received 2L therapy, and 35% received 3L therapy. Androgen receptor pathway inhibitor (ARPI) was the most common Rx in the 1L setting (73.7%), followed by taxane (10%) and sipuleucel-T (3.8%). In 2L, ARPI was again the most frequent Rx (46.8%), followed by taxane (23.6%). In 3L, taxane became the most frequent Rx (35.4%), followed by ARPI (26.2%) and radium-223 (4.9%). Rx trends per year will be presented in the meeting. Conclusions: In the current era,a high proportion of pts with mCRPC receiving 1L Rx do not receive a subsequent line of Rx (39% of pts do not receive 2L, and 65% do not receive 3L Rx). ARPIs and taxanes remain the most frequently used Rx options in most L of therapy. These findings highlight the need for better tolerated Rx, therapies with a novel mechanism of action, and improved access to care for our pts. Rx trends per year will be presented in the meeting. Rx patterns in pts with mCRPC. Rx 1L, n (%)N = 12,333 2L, n (%)N = 7475 3L, n (%)N = 4316 4L, n (%)N = 2339 5L, n (%)N = 1225 ARPI 9085 (73.7) 3503 (46.8) 1130 (26.2) 356 (15.2) 157 (12.8) PARPi-based therapies 93 (0.8) 185 (2.5) 142 (3.3) 98 (4.2) 47 (3.9) Platinum-based therapy 127 (1) 216 (2.9) 198 (4.6) 148 (6.3) 130 (10.6) Lu-177-based therapies 50 (0.4) 109 (1.5) 169 (3.9) 143 (6.3) 103 (8.4) Radium-223 183 (1.5) 241 (3.2) 213 (4.9) 125 (5.3) 79 (6.4) Sipuleucel-T 474 (3.8) 94 (1.3) 53 (1.2) 18 (0.8) 8 (0.7) Taxane 1240 (10) 1763 (23.6) 1526 (35.4) 935 (40) 393 (32) Pembrolizumab 24 (0.2) 41 (0.5) 36 (0.8) 27 (1.2) 29 (2.4) Other*/not-applicable** 665 (5.4)/392 (3.2) 1000 (13.4)/323 (4.3) 602 (14)/247 (5.7) 320 (13.7)/169 (7.2) 162 (13.3)/117 (9.5) *Includes drugs approved for mCRPC in combination with unapproved agents or clinical trial drugs. **Includes agents not approved for prostate cancer.

95 Background: Most pts with prostate cancer with bone metastasis experience symptomatic skeletal events (SSEs) [PMID: 23884473], leading to increased morbidity and mortality. BMAs have been approved since 2007 and are recommended in mCRPC to prevent SSEs in pts with bone metastases or a high risk of osteoporotic fracture. Herein, we sought to assess the real-world trends of BMA administration in pts with mCRPC. Methods: We used the de-identified nationwide Flatiron Health electronic health record (EHR)-derived database to extract pt-level data. Inclusion criteria: pts with mCRPC with available date of diagnosis of mCRPC and information on receipt of BMA. Pts were categorized into 2 cohorts based on BMA receipt (e.g., bisphosphonates, denosumab, teriparatide, romosozumab) or not. Treatment trends of BMA by year of mCRPC diagnosis were summarized using frequency and percentages. Baseline characteristics at the time of mCRPC diagnosis (age, race-ethnicity, practice type, insurance) were compared between cohorts using the Wilcoxon rank-sum and Chi-squared tests. Results: Of 24,105 pts with metastatic prostate cancer in the dataset, 14,112 with mCRPC were eligible and included. 7,990 (56.6%) received BMA while 6,122 (43.4%) did not. In BMA receipt cohort: median age was 75 (IQR 67 – 81), majority were White (61%), treated in community practice (87%) and had commercial health plan (80%). In BMA non receipt cohort: median age was 75 (IQR 68 – 82), majority were White (61%), treated in a community practice (78%), and had commercial health plan (76%). The use of BMA decreased over time from 56% in 2013 to 46% in 2024 (Table). Statistically significant differences existed between cohorts in race-ethnicity, practice type, and insurance (p < 0.001) and will be presented at the meeting. Conclusions: Despite recommendations to use BMAs in pts with mCRPC and bone metastasis or high risk of osteoporotic fracture, our findings reveal a low utilization rate of BMAs in mCRPC setting with a notable decline over time. This highlights the need to incorporate the use of BMAs in clinical practice and improve access to these agents. Trends by year of mCRPC diagnosis in receipt of the first BMA and frequency and percentage of each agent. Year 2013N = 549 2014N = 1035 2015N = 1223 2016N = 1345 2017N = 1441 2018N = 1346 2019N = 1469 2020N = 1380 2021N = 1352 2022N = 1393 2023N = 1196 2024N = 383 Receipt of BMA, n (%) 316 (56) 674 (65) 792 (65) 837 (62) 886 (61) 766 (57) 816 (56) 766 (56) 692 (51) 713 (51) 556 (46) 176 (46) Denosumab, n (%) 190 (60.1) 450 (66.8) 557 (70.3) 599 (71.6) 606 (68.4) 533 (70) 526 (64.5) 488 (63.7) 410 (59.2) 414 (58.1) 290 (52.2) 94 (53) Zoledronic acid, n (%) 120 (38) 220 (32.6) 228 (28.7) 233 (27.8) 270 (30.5) 225 (29) 281 (34.4) 273 (35.6) 277 (40) 295 (41.4) 258 (46.4) 82 (47) Teriparatide, other bisphosphonates, n (%) 6 (1.9) 4 (0.6) 7 (1) 5 (0.6) 10 (1.1) 8 (1) 9 (1.1) 5 (0.7) 5 (0.7) 4 (0.5) 8 (1.4) 0

90 Background: Poly(ADP) ribose polymerase inhibitors and pembrolizumab are approved for patients with metastatic prostate cancer (mPC) with susceptible genomic alterations based on NGS testing. We recently reported that the rate of NGS testing in mPC remains low at 29.3% in the United States (1). Herein, we investigated the rates of NGS testing at the end of life in a real-world patient population. Methods: This study utilized the nationwide Flatiron Health electronic health record (EHR)-derived de-identified database. Eligibility: diagnosis of mPC, with information on receipt of NGS testing (blood/tissue) and recorded date of death. The time between each patient’s first NGS testing result date and date of death was measured and categorized into 3 groups: NGS test results delivered more than 3 months (mo) before death, within 3 mo of death, and delivered after death. Frequencies and percentages of these 3 categories were reported, also by the year of death. Results: Out of 24,105 patients with mPC in the database, 3,397 had both a recorded date of death and underwent NGS testing and were eligible and included. The median age was 74 (IQR 67–80), 70% were White non-Hispanic, and 83% were treated in a community-based practice. In these patients, 2,901 (85.4%) NGS results were reported more than 3 mo before death, 457 (13.5%) within 3 mo of death, and 39 (1.1%) after death. From 2015 to 2024, the percentage of patients receiving their result more than 3 mo before death increased from 27% to 91%, while those receiving it within 3 mo before death decreased from 55% to 8.5%, and the percentage of NGS results reported after death declined from 18% to 0.6%. Baseline characteristics (including race-ethnicity, insurance plan, practice type [academic vs. community]) in these categories will be presented at the meeting. Conclusions: Despite the availability of life-prolonging targeted therapies based on NGS testing results for patients with mPC, in ∼14% of patients, the first NGS testing results were reported within 3 mo before death or after death. The lack of timely NGS testing limits patients' access to life-prolonging personalized treatments in earlier settings, resulting in missed opportunities for improved outcomes. 1. Hage Chehade, Swami, JAMA Network Open, 2024.

866 Background: Emerging data suggests that Language Models (LLMs) provide an unprecedented opportunity to extract critical data from the unstructured pathology reports. However, the performance of state of art (SOTA) models on clinically meaningful tasks such as assessment of pathologic responses in patients receiving neoadjuvant chemotherapy (NAC) in muscle invasive bladder cancer(MIBC) is not well known. Methods: This retrospective cohort included patients with pathologically confirmed Muscle-Invasive Bladder Cancer from the year 2018 to 2024. Selected patients included those who underwent Neoadjuvant chemotherapy followed by definitive cystectomy at Mayo Clinic. Gold standard labels were manually curated by trained clinicians. The state of the art (SOTA) LLM – GPT4 – was utilized using a structured zero-shot prompt to extract relevant phenotypic variables such as tumor site, tumor histology and additional histological variants if any, presence of in-situ / invasive carcinoma, presence of lamina propria / muscularis propria / lympho-vascular / perineural invasion, and T-stage from unstructured pathology reports. Structured prompts were iteratively developed for each data variable and validated using ~5% of the total dataset. The extracted variables were manually compared and labelled as True positive / True negative / False positive / False negative. Final performance was assessed against a held-out expert annotated test dataset using evaluation metric (accuracy). Results: A total of 200 reports from 99 patients were extracted by the LLM. Of the 200 reports, 41 duplicate / other pathology reports (e.g. – autopsy, cholecystectomy, etc.) were removed during final assessment. Significant characteristics such as tumor site, histology, and additional histology variant had an accuracy of 89%, 87%, and 91% respectively. Notably, accuracy of detection of in-situ versus invasive carcinoma was 70% versus 91%. Specific variables such as lamina propria / muscularis propria invasion / T-stage / lympho-vascular invasion / perineural invasion were answered with the accuracy of 75%, 75%, 67%, 90%, and 94% respectively. Conclusions: LLMs have the potential to extract critical data from unstructured pathology reports and automate assessment of pathologic responses in MIBC patients receiving NAC. Iterative improvements in performance can be achieved by improving prompt design using expert clinical input. This process can be potentially scaled to automate cancer registries.