Jajja, Salman Ayub

5108 Background: We previously proposed a hybrid framework combining LLMs and rule based algorithm (RBA) for automating risk stratification in locPCa. Herein, we aim to validate the risk stratification agent (RSA) in a prospective cohort, develop & evaluate the treatment recommendation agent (TRA) based on NCCN guidelines, and develop an interactive interface to facilitate clinicians for accurate risk stratification and treatment recommendations at the point of care. Methods: This study included pts with locPCa (2004-2024) presenting at Mayo Clinic with at least 1 positive prostate biopsy and MRI report available. For RSA prospective validation, GPT4 extracted key phenotypic variables (PSA, T stage, prostate volume, number of cores, Gleason patterns, grade group) from unstructured MRI and biopsy reports using a zeroshot prompt. An RBA then classified pts into NCCN risk groups. The agent performance was compared with the treating clinician's documentation and evaluated against gold-standard labels manually annotated by two independent clinicians. For development of TRA, two experiments were performed using GPT4 with and without retrieval-augmented generation (RAG) to generate treatment plans. Generated treatment plans were evaluated using NCCN guideline-informed treatment decision tree based algorithm (DTA). Evaluation metrics, weighted – accuracy (acc) and F1, were computed. A clinician facing interface (lisr.org/risk) was developed to provide accurate risk stratification and treatment plans. Results: A total of 858 pts were included (500 for prospective validation, 358 for treatment recommendations). Prospective validation for RSA demonstrated a higher F1 score of 0.89 compared to the treating clinician (F1: 0.58). Treatment recommendation experiments showed that GPT4 with RAG achieved higher acc (64% full – all correct treatment options and 36% partial – at least 1 correct treatment option) compared to GPT4 alone (35% and 65%, respectively). Sensitivity analysis using DTA-informed GPT4 note generation achieved 94% full acc and 6% partial acc. GPT4 alone generated hallucinated treatment options in 71% of cases, while GPT4 with RAG reduced this to 32%. Conclusions: The multi-agent framework based on LLM and RBA achieves high accuracy in risk stratification and treatment recommendation for locPCa. A multi-agent framework with an interactive interface holds high promise to enable efficient, accurate decision-making, and improve locPCa management at the point of care Performance evaluation. RSA(acc; F1) % TRA(full acc; partial acc) % Clinician RSA* GPT4 RAG+GPT4 DTA+GPT4 Overall 60; 58 89; 89 29; 71 64; 36 94; 6 V. High 94; 33 96; 96 55; 45 69; 31 93; 7 High 46; 52 92; 89 - - - Int. Unf 80; 61 91; 90 5; 95 38; 62 93; 7 Int. Fav 57; 64 84; 87 0; 100 98; 2 100; 0 Low 57; 65 87; 80 100; 0 95; 5 90; 10 V. Low 8; 13 50; 67^ - - - *Statistical significance (p<0.01) except for ^V. Low.

e17155 Background: Localized PCa is classified into six NCCN risk categories, each with specific treatment recommendations. However, treatment concordance across these categories and its variation by race in clinical practice remain unexplored. Therefore, we aim to describe patterns of guideline-concordant treatment receipt for localized PCa across risk groups and race. Methods: Surveillance, Epidemiology, and End Results (SEER) database (2018–2020) was queried to identify patients with localized PCa from 17 registries. Data regarding clinical T stage, Gleason patterns, PSA level, biopsy cores and initial treatment received (radiation, radical prostatectomy, systemic therapy) was obtained. Each patient was then stratified into one of six NCCN-based risk categories using a rule-based algorithm. Using the stratified risk category of each patient, treatment recommendations were obtained from the NCCN guidelines. Concordance was assessed by comparing the received treatment and guideline-recommended treatment. A focused bi-variate logistic regression analysis was performed adjusting for risk groups, and race. Odds ratio (OR) with 95% confidence intervals (CI) were computed to quantify the magnitude of guideline-discordant treatment receipt. Results: This analysis included 69,584 patients with localized PCa (White: 53,875 [77.4%]; Black: 11,265 [16.2%]; Asian/Pacific Islander: 4,133 [5.9%]; American Indian/Alaska Native: 311 0.5%]). In terms of risk groups, patients in higher risk categories had increased odds of discordance compared to the lower risk groups (Table). In terms of race, Black men had significantly higher odds of receiving guideline-discordant therapy (OR:1.67, 95% CI:1.58–1.76), while no statistically significant difference was observed with American Indian/Alaska Native men (OR:1.21, 95% CI: 0.92–1.61) and Asian/Pacific Islander men (OR:1.05, 95% CI:0.97–1.14) compared to White men. Conclusions: Population level data, based on limited sample size in some groups, suggests that treatment concordance with NCCN guidelines varies by risk group and race in localized PCa. Black patients, and those with higher risk groups were more likely to receive guideline-discordant therapy. These findings highlight the need for strategies and prospective studies at individual patient level to evaluate and enhance guideline adherence to promote personalized care for these patients. Variable Discordant Treatment (%) OR (95% CI) White 22168 (41.1) REF Black 5653 (47.5) 1.67 (1.58-1.76) * Asian/Pacific Islander 1822 (44.1) 1.05 (0.97-1.14) American Indian/Alaska Native 191 (61.4) 1.22 (0.92- 1.61) Very Low Risk 770 (9) REF Low Risk 146 (2) 0.16 (0.13-0.19) * Favorable Risk 35 (0.4) 0.03 (0.02-0.04) * Unfavorable Risk 15125 (83.2) 36.30 (33.72-39.25) * High/Very High Risk 13441 (50) 7.32 (6.83-7.86) * * p <0.05.

e23101 Background: Liver cell carcinoma (LCC) and hepatic failure (HF) are major contributors to global adult mortality. Identifying trends and disparities in LCC and HF-related deaths is critical for addressing public health challenges and guiding prevention strategies. This study examines national trends in LCC and HF-related mortality from 1999 to 2023, with an emphasis on demographic, geographic, and temporal disparities in age-adjusted mortality rates (AAMRs). Methods: Using the CDC WONDER database (1999–2023), this descriptive study identified deaths attributed to LCC (ICD-10 code C22.0) and HF (ICD-10 code K72.9) among adults aged 25 and older. Mortality data were stratified by gender, race/ethnicity, census region, urban-rural classification, and state. Age-adjusted mortality rates (AAMRs) per 1,000,000 individuals were calculated, and trends were analyzed using Joinpoint regression to determine annual percent changes (APCs) with 95% confidence intervals (CIs). Statistical significance was set at p < 0.05, and the study followed STROBE guidelines. Results: From 1999 to 2023, there were 22,963 deaths attributed to LCC and HF (Men: 17,822; Women: 5,141). Overall, AAMRs declined from 1999 to 2007 (APC: -2.4), stabilized between 2007 and 2017 (APC: 0.2), and declined again from 2017 to 2023 (APC: -3.5). Men experienced a steady AAMR decline from 1999 to 2010 (APC: -2.1), followed by stabilization (2010–2017, APC: 0.7), and further decline (2017–2023, APC: -4.4). Women showed a sharp AAMR decrease from 1999 to 2006 (APC: -3.8) and a relatively stable trend thereafter (APC: -0.1). Non-Hispanic Blacks people experienced an overall AAMR decline from 1999 to 2020 (APC: -0.9). Among Non-Hispanic Whites people, AAMRs decreased initially (1999–2006, APC: -2.6), rose slightly (2006–2018, APC: 0.5), and declined further (2018–2020, APC: -7.3). Regional AAMRs varied, with the West having the highest overall AAMR (5.6; 95% CI: 5.4–5.7) and the Midwest the lowest (3.2; 95% CI: 3.1–3.3). Hawaii reported the highest state AAMR (10.3; 95% CI: 8.9–11.6), while Arkansas had the lowest (2.4; 95% CI: 2.0–2.9). Urban-rural disparities revealed a sharper AAMR decline in metropolitan areas (1999–2005, APC: -3.9) and the non-metropolitan areas showed a slight increase (1999–2020, APC: 1.3). Conclusions: LCC and HF-related mortality rates have shown significant declines over the past two decades, with variations across gender, race, and geographic locations. Men and urban populations experienced steeper reductions in AAMRs, while rural populations exhibited slower improvements or slight increases in mortality. These findings highlight the need for targeted public health interventions to address disparities and further reduce LCC and HF-related mortality in vulnerable populations.

e16157 Background: The optimal treatment for resectable gastric/GEJ AC is unclear due to limited direct comparisons among chemotherapy and chemoradiation regimens. Addressing this gap is critical to improve patient outcomes. We performed this network meta-analysis (NMA) to determine the relative efficacy of multimodality regimens for resectable gastric/GEJ AC. Methods: MEDLINE, EMBASE, Scopus, and CENTRAL were searched from inception till September 20, 2024, to identify phase III trials evaluating perioperative/neoadjuvant systemic therapy ± radiation in resectable gastric/GEJ AC. Primary outcomes were disease-free survival (DFS) and overall survival (OS). A frequentist NMA was conducted to compare hazard ratios (HR) and 95% CI for each outcome. Relative treatment rankings were assessed using P-scores. Results: Fifteen trials (8,072 patients) were analyzed. pFLOT (perioperative fluorouracil, leucovorin, oxaliplatin, and docetaxel) ranked highest for DFS in the overall population (P score 91%) (Table), significantly outperforming surgery alone (HR 0.48, 95% CI 0.38-0.60) and nCROSS (neoadjuvant paclitaxel, carboplatin and radiotherapy) (0.67, 0.56-0.81). For OS, nPLF+nCRT(EP) (neoadjuvant cisplatin, leucovorin and fluorouracil for induction, followed by etoposide, cisplatin, and radiotherapy) (P score 90%) and pFLOT (P score 87%) were the top regimens, with pFLOT significantly outperforming surgery alone (0.57, 0.45-0.72), and nCROSS (0.73, 0.60-0.89). Adding neoadjuvant chemoradiation to pFLOT provided no additional OS (1.14, 0.76-1.72) or DFS (1.22, 0.83-1.82) benefit. Similarly, adding pembrolizumab to perioperative chemotherapy (CF/CX) (cisplatin and fluorouracil/capecitabine) was not superior to pFLOT in terms of DFS (1.10, 0.71-1.69) or OS (1.09, 0.69-1.72). Conclusions: pFLOT demonstrated superior efficacy in resectable gastric/GEJ AC, surpassing surgery alone, nCROSS, and other perioperative regimens. The additive role of immunotherapy to pFLOT requires further investigation to refine patient selection and optimize outcomes. Regimen Overall Population GEJ cohort OS DFS OS DFS nPLF+nCRT(EP) 1 (90) 2 (84) 1 (85) NA pFLOT 2 (87) 1 (91) 2 (79) 2 (88) pPembrolizumab+CF/CX 3 (75) 3 (81) 5 (41) - nECX 4 (61) 5 (55) - - pCF 5 (59) 8 (51) 3 (72) - pECF/ECX 6 (47) 6 (55) 7 (40) 3 (57) nECX+aCRT(CX) 7 (45) 7 (52) 9 (31) - nCF 8 (42) 10 (28) 6 (40) - nCROSS 9 (41) 9 (35) 8 (34) 4 (43) pECF/ECX/FLOT+nCRT(F/X) 10 (38) 4 (57) 4 (72) 1 (90) nCRT(CF) 11 (9) 11 (6) - 5 (14) Surgery 12 (7) 12 (5) 10 (5) 6 (9) Data reported as rank (P-score). Abbreviations: n: neoadjuvant; p: perioperative; ECX: epirubicin + cisplatin + capecitabine; CF: cisplatin + fluorouracil; ECF: epirubicin + cisplatin + fluorouracil; aCRT(CX): adjuvant radiotherapy + cisplatin + capecitabine; CRT(F/X): fluorouracil/capecitabine + radiotherapy; CRT(CF): cisplatin + fluorouracil + radiotherapy.

e16418 Background: Pancreatic cancer remains one of the most fatal malignancies globally, posing a significant burden on older populations. Despite numerous reports on its prevalence, a detailed comparative analysis of national trends among adults aged 70 and older from 1990 to 2021 is scarce. This study seeks to bridge this gap by evaluating prevalence rates using data from the Global Burden of Disease (GBD) study. Methods: Prevalence data for pancreatic cancer in individuals aged 70 years and older were obtained from the GBD-2021 dataset, encompassing 204 countries and territories for the years 1990 and 2021. The Estimated Annual Percent Change (EAPC) and corresponding 95% confidence intervals (CI) were calculated to assess trends over the period from 1990 to 2021. Results: The prevalence of pancreatic cancer has shown a significant increase since 1990, reaching its highest levels in 2021. In 1990, the United States of America (USA) reported the highest prevalence, followed by Japan and China. However, by 2021, Japan ranked first in prevalence, followed by China and the USA. The analysis revealed that in 1990, the highest prevalence rates of pancreatic cancer were observed in Greenland (91.04 per 100,000), followed by Finland (82.57 per 100,000) and Japan (76.30 per 100,000). By 2021, Japan reported the highest prevalence (126.73 per 100,000), followed by Finland (119.20 per 100,000) and Canada (109.22 per 100,000). Despite the observed prevalence rates from 1990 to 2021, Turkmenistan exhibited the most significant increase (EAPC: 15.41, 95% CI: 10.91, 21.41), followed by the Republic of Cabo Verde (EAPC: 13.61, 95% CI: 8.67, 19.59) and Mongolia (EAPC: 4.81, 95% CI: 3.11, 7.20). In contrast, the most significant declines were recorded in the Republic of San Marino (EAPC: -0.308, 95% CI: -0.51, -0.03), followed by Guam (EAPC: -0.24, 95% CI: -0.37, -0.08) and Cuba (EAPC: -0.21, 95% CI: -0.33, -0.09). Conclusions: The prevalence of pancreatic cancer among adults aged 70 and older has increased significantly over the past three decades, with distinct regional patterns. Countries like Japan, China, and the USA continue to report the highest prevalence rates. These findings highlight the urgent need for targeted prevention, screening, and treatment strategies, particularly in high-burden regions, to mitigate the growing impact of pancreatic cancer in aging populations. Prevalence percent changes. Metric Region 1990 2021 EAPC with 95% CI (1990-2021) Highest prevalence percent change Turkmenistan 0.62 10.16 15.41 (10.91, 21.41) Republic of Cabo Verde 3.23 47.18 13.61 (8.67, 19.59) Mongolia 5.25 30.53 4.81 (3.11, 7.20) Lowest prevalence percent change Republic of San Marino 52.61 36.36 -0.308 (-0.51, -0.03) Guam 16.96 12.87 -0.24 (-0.37, -0.08) Cuba 32.65 25.52 -0.21 (-0.33, -0.09)

e22572 Background: Pancreatic cancer is a highly lethal malignancy that disproportionately impacts older populations, contributing significantly to global disease burden. Disability-adjusted life years (DALY) serve as a comprehensive metric to quantify the combined impact of mortality and morbidity. This analysis of the Global Burden of Disease Study-2021 (GBD-2021) aims to characterize trends in DALY rates attributable to pancreatic cancer among individuals aged 70 years and older across 204 nations. Methods: Data on DALY associated with pancreatic cancer for individuals aged 70+ were extracted for 204 nations from the GBD-2021 study for the years 1990 and 2021. Estimated annual percent change (EAPC) with 95% confidence intervals (CI) were calculated from 1990 to 2021. Results: The DALY rates were accessed among individuals aged 75 and older diagnosed with pancreatic cancer. In 1990, Greenland exhibited the highest DALY rate globally (2825.65), but by 2021, this position was surpassed by the United Arab Emirates (2775.52), with Greenland declining to the second-highest rate (2082.43). On the other hand, Turkmenistan had the lowest DALY rates in 1990, but by 2021, Mozambique had taken this position, with Turkmenistan ranking 170th. Despite the observed DALY rates from 1990 to 2021, Turkmenistan experienced the highest change (EAPC: 15.23, 95% CI: 10.88–21.18), followed by the Republic of Cabo Verde (EAPC: 12.88, 95% CI: 8.14–18.75) and Mongolia (EAPC: 4.53, 95% CI: 2.90–6.88). A decrease in DALY was observed in over 24 nations, with the Republic of San Marino recording the largest decline (EAPC: -0.37, 95% CI: -0.55, -0.12), followed by Bermuda (EAPC: -0.26, 95% CI: -0.43, 0.03) and Greenland (EAPC: -0.26, 95% CI: -0.45, -0.009). Conclusions: From 1990 to 2021, DALY rates related to pancreatic cancer among individuals aged 70 and older rose worldwide, particularly in Turkmenistan, the Republic of Cabo Verde, and Mongolia. There is a pressing need for global adoption of screening programs to enable early detection and treatment of these cases. Metric Region 1990 2021 EAPC with 95% CI (1990-2021) HIGHEST DISABILITY-ADJUSTED LIFE YEARS (DALYS) PERCENT CHANGE Turkmenistan 19.66 319.09 15.22 (10.88–21.18) Republic of Cabo Verde 98.12 1361.76 12.88 (8.14–18.75) Mongolia 172.14 952.05 4.53 (2.90–6.88) LOWEST DISABILITY-ADJUSTED LIFE YEARS (DALYS) PERCENT CHANGE Republic of San Marino 1323.96 834.01 -0.37 (-0.55, -0.12) Bermuda 1696.97 1249.11 -0.26 (-0.43, 0.03) Greenland 2825.66 2082.43 -0.26 (-0.45, -0.009)

Abstract Background Optimal treatment for resectable gastric and gastroesophageal junction (GEJ) adenocarcinoma remains unclear due to limited head-to-head comparisons among chemotherapy and chemoradiation regimens. This network meta-analysis aimed to determine the relative efficacy of available multimodality treatment regimens in these patients. Methods MEDLINE, EMBASE, Scopus, Web of Science, and CENTRAL were searched till September 20, 2024. Phase 3 randomized trials evaluating perioperative/neoadjuvant systemic therapy ± radiation in resectable gastric/GEJ adenocarcinoma were included. Primary outcomes were disease-free survival (DFS) and overall survival (OS). Frequentist network meta-analysis was conducted to compare hazard ratios (HRs) and 95% confidence intervals (CIs). Results Fifteen trials (8072 patients) were analyzed. pFLOT (perioperative fluorouracil, leucovorin, oxaliplatin, and docetaxel) ranked highest for DFS in the overall population (P-score 91%), achieving significant improvements compared to surgery alone (HR 0.48, 95% CI, 0.38-0.60) and nCROSS (neoadjuvant paclitaxel, carboplatin, and radiotherapy) (0.67, 0.56-0.81). For OS, nPLF + nCRT(EP) (neoadjuvant cisplatin, leucovorin, and fluorouracil for induction, followed by etoposide, cisplatin, and radiotherapy) (P-score 90%) and pFLOT (P-score 87%) were the top regimens. pFLOT significantly outperformed surgery alone (0.57, 0.45-0.72) and nCROSS (0.73, 0.60-0.89). Based on the limited data available, adding neoadjuvant chemoradiation to pFLOT provided no additional OS (1.14, 0.76-1.72) or DFS (1.22, 0.83-1.82) benefit. Similarly, adding pembrolizumab to perioperative chemotherapy (cisplatin and fluorouracil/capecitabine) was not superior to pFLOT (DFS: 1.10, 0.71-1.69; OS: 1.09, 0.69-1.72). Conclusions pFLOT demonstrated superior efficacy in resectable gastric/GEJ adenocarcinoma, outperforming surgery alone, nCROSS, and alternative perioperative regimens. The additive role of immunotherapy requires further investigation to optimize patient selection and outcomes.

270 Background: Inconsistent reporting of subgroup data across clinical trials makes interpretation and application of the evidence from trials challenging. We conducted a living systematic review to summarize evidence for American Society of Clinical Oncology (ASCO) guidelines for management of metastatic castration resistant prostate cancer (mCRPC). Here, we provide an overview of reporting patterns of subgroup data across mCRPC trials. Methods: MEDLINE and EMBASE were systematically searched from each database's inception through September 20 th , 2024, to identify phase II/III/IV randomized clinical trials assessing the efficacy and safety of treatment options in mCRPC. For outcomes of overall survival (OS) and progression free survival (PFS), data from the subgroup analyses was collected and compared across trials to assess heterogeneity in the reporting of subgroups. Results: This analysis included 141 trials (phase II: 83; phase III: 56; phase IV: 2). OS was reported by 113 (80%) trials; 32 (28%) reported OS by any subgroups with 95 unique subgroup categories. PFS was reported by 110 (78%) trials; 23 (21%) reported PFS by any subgroups with 63 unique subgroup categories. Even among consistently reported subgroups, such as age, ECOG performance status, race, geographical area, and Gleason score, considerable heterogeneity in different subgroup levels/thresholds was observed across trials. Top ten most commonly reported subgroups are summarized (Table). Conclusions: Lack of consistent reporting for subgroup analysis in mCRPC trials preclude meaningful conclusions about clinically relevant subgroup and synthesizing evidence for clinical guidelines. A standardized framework for consistent reporting of subgroup data across trials can help inform clinical practice and clinical practice guidelines. OS PFS Number of trials 113 110 Reporting subgroup analysis – N (%) 32 (28) 23 (21) Commonly reported subgroups – (N) Age (26)ECOG-PS (26)LDH (21)PSA (18)Geographic area (15)Alkaline phosphatase (14)Visceral disease (14)Gleason score (9)Race (9)Brief pain inventory level (8) Age (21) ECOG-PS (19)PSA (18)LDH (14)Geographic area (13)Visceral disease (11)Gleason score (10)Alkaline phosphatase (9)Bone metastasis (9)Disease location (9) Age – (N) <65 (14); ≥65 (9); ≤68 (1); ≥69 (1); <70 (5); ≥70 (5); ≤74 (4); 65-75 (5); ≤ 71 (1); >71 (1); ≥75 (11); >Median (1); <Median (1) <65 (12); 65-74 (6); ≥65 (6); <70 (5); ≥70 (5); <75 (4); ≥75 (10); 76-80 (1); >80 (1) ECOG-PS – (N) 0 (15); 0-1 (11), 1 (11); 1-2 (5); 2 (12) 0 (15); 0-1 (5), 1 (12); 1-2 (2); 2 (6) Race – (N) White (9)Non-white (6)African American/black (2)Asian (2) White (8)Non-white (5)African American/black (2)Asian (4) Geographic Area – (N) Europe (8)North America (14)United States (1)Australia (1)Asia (2) Europe (11)North America (13)United States (1)Australia (1)Asia (2)North and South America (2) Gleason Score – (N) 2-6 (1); 7 (1); 8 (1); <8 (8); ≥ 8 (8); 9-10 (1) <8 (10); ≥ 8 (10)

745 Background: The combination of EV-P therapy has recently been approved as first-line (1L) therapy for patients with LA/mUC based on the results of the EV-302 trial. Herein, we report the updated results from a cohort of patients with LA/mUC who received EV-P in real world clinical setting. Methods: This retrospective study included patients with LA/mUC (07/2022-08/2024) at the Mayo Clinic who completed at least one cycle of EV-P. The best overall response (BOR) was evaluated using radiographic imaging and was adjudicated as complete response (CR), partial response (PR), stable disease (SD), mixed response (MR), or progressive disease (PD). The median progression-free survival (PFS), defined as the time from treatment initiation to disease progression or death, was estimated using the Kaplan-Meier method. Patients who had not experienced disease progression by their last follow-up were censored for the PFS analysis. Results: A total of 120 patients were included; 79 (65.8%) were males, 41 (34.1%) females, 109 (91%) White, and 119 (99%) were non-Hispanic/Latino. The median age was 72 years (IQR: 65-77). Of these, only 22 (18.3%) had locally advanced disease while the remaining had mUC; 62 (51.6%) had lower tract urothelial carcinoma (LUTC), 36 (30%) had upper tract urothelial carcinoma (UTUC). The median follow-up time was 7.1 months (IQR: 4.93-9.26). In terms of BOR, 56 patients (46.6%) experienced PR, 34 (28.3%) CR, 12 (10%) PD, 7 (5.8%) MR, and 8 (6.6%) experienced SD. 3 (2.5%) patients were not evaluable. The BOR across subgroups, LA and mUC, including LUTC and UTUC, is summarized in the table. Among the responders, the median number of treatment cycles to first response was 3 (IQR: 2.25–4). Within the follow-up, 28 patients (23.3%) experienced eventual radiographic disease progression; death was observed in 25 patients (20.8%). The median PFS was 12.7 months (95% CI: 9.8-NE), and OS was 25.1 months (16.3-NE). Conclusions: In real world setting, the combination of EV-P in LA/mUC demonstrated clinical efficacy compatible with the findings from the EV-302 trial. Our results showed comparable efficacy with EV-P in both UTUC and LUTC, highlighting it's broad therapeutic potential. However, clinical data at longer follow-up is required to fully evaluate overall survival and the durability of response. mUC LA Overall LUTC UTUC Total patients 98 62 36 22 BOR CR 23 (23.4%) 13 (20.9%) 10 (28.5%) 11 (50%) PR 49 (52.6%) 31 (50%) 18 (50%) 7 (31.8%) SD 6 (6.4%) 2 (3.2%) 4 (11.4%) 2 (9.0%) PD 10 (10.7%) 9 (15.5%) 1 (2.8%) 2 (9.0%) MR 7 (7.5%) 5 (8.6%) 2 (5.7%) 0 (0%)

136 Background: The ECOG performance status (PS) is a critical measure for evaluating functional capacity and determining clinical trial eligibility in patients with advanced prostate cancer. We aim to assess the capability of large language models (LLMs) to accurately infer ECOG PS from unstructured oncology notes. Methods: This retrospective study included patients with metastatic castration-resistant prostate cancer (mCRPC) receiving Lutetium-177–PSMA-617 ( 177 Lu) at the Mayo Clinic (2022-2023). Baseline ECOG PS scores were manually curated by a trained clinician from clinical notes recorded within four weeks prior to 177 Lu initiation. Structured zero-shot prompts were then used to prompt GPT-4 to infer ECOG PS scores from unstructured oncology notes. Total dataset was randomly sampled to prompt development (20%) and held-out test set (80%). Three experimental setups were employed to assess model performance: (i) utilizing unmasked clinical notes, (ii) clinical notes with only numerical ECOG PS scores masked while retaining narrative descriptive/assessment by oncologist of ECOG score, and (iii) clinical notes with both numerical scores and narrative ECOG descriptors masked. Final performance was assessed using evaluation metrics (precision, recall, F1 score) on the held-out test set. Results: A total of 240 patients (n: 50 prompt development; n: 190 test set) were included in the analysis. The median age was 70 (IQR: 66-76); a majority of patients were White (n: 229; 95%) and non-Hispanic (n: 233; 97%). The most prevalent ECOG PS score was 0 (n: 140; 58%), followed by 1 (n: 88; 37%), and 2 (n: 12; 5%). The performance was numerically higher when using unmasked notes (weighted average precision: 0.89; recall: 0.89; F1: 0.89), compared to using notes with masked scores only (weighted average precision: 0.76; recall: 0.79; F1: 0.75) and notes with both scores and narrative masked (weighted average precision: 0.57; recall: 0.53; F1: 0.48). Metrics by each ECOG PS score is outlined (Table). Conclusions: Large language models demonstrate significant potential in precisely extracting ECOG performance status from unstructured oncology notes, offering valuable support for treatment decision-making and clinical trial enrollment. However, model performance declines as reliance on implicit reporting within clinical notes increases. Future efforts focusing on prediction of ECOG using a series of notes over time will be the next step. ECOG Unmasked oncology notes ECOG Score masked only ECOG Score + ECOG narrative marked Precision Recall F1 Score Precision Recall F1 Score Precision Recall F1 Score 0 0.89 0.92 0.91 0.75 0.93 0.83 0.59 0.74 0.66 1 0.88 0.86 0.87 0.76 0.74 0.75 0.19 0.47 0.27 2 1.00 0.82 0.90 0.78 0.33 0.47 0.78 0.16 0.26