Naqvi, Syed Arsalan Ahmed

112 Background: We now have more than a decade of experience using Androgen Receptor Pathway Inhibitors (ARPIs) across various disease states of prostate cancer, and the safety of their combination with Lutetium-177–PSMA-617 has been demonstrated in VISION and other large randomized clinical trials. However, real-world data on treatment patterns and outcomes involving combination therapies with 177 Lu-PSMA-617 remain limited. Methods: For this analysis, we utilized the Mayo Clinic Rochester radiopharmaceutical database, a prospectively maintained retrospective database containing all patients who received 177 Lu-PSMA-617 at our institution. We focused on patients who started treatment from March 2022 to March 2023. Patients receiving an ARPI (abiraterone acetate, enzalutamide, apalutamide, or darolutamide) during the course of 177 Lu-PSMA-617 treatment were identified, inclusive of patients who continued using an ARPI prescribed as a prior line of treatment and those who started (i.e., switched) to a new ARPI. Baseline clinicopathologic and imaging characteristics were abstracted and compared using the Mann-Whitney U and Chi-square tests. Best PSA response during treatment was reported as a percent decline from baseline. Survival was calculated from the date of the first cycle of 177 Lu-PSMA-617. PSA50 response and overall survival (OS) outcomes for the two groups (concurrent use of ARPI vs. 177 Lu-PSMA-617 alone) were compared using the Chi-square test and Kaplan-Meier method, respectively. A multivariate Cox regression analysis was performed, including established prognostic factors. Results: An ARPI was prescribed to 106 of the 256 patients (41.4%) starting 177 Lu-PSMA-617 in the interval of March 2022 to March 2023. With regards to baseline characteristics, those receiving an ARPI concurrently with 177 Lu-PSMA-617 had a lower PSA (3.4 vs 29.7 ng/mL, p < 0.001) and a lower frequency of bone (77.4% vs. 87.4%, p = 0.035) and visceral metastases (18.9 vs 34%, p = 0.008) at start of treatment. Median follow-up for the overall cohort (IQR) was 19.1 months (8.7 – 23.6). Patients receiving an ARPI plus 177 Lu-PSMA-617, as compared to 177 Lu-PSMA-617 alone, were more likely to complete all 6 planned doses of treatment (63.2% vs. 48.7%, p < 0.001), though the PSA50 response rate was similar (49.1% vs 47.3%, p = 0.786). Median OS [95% CI] for the overall cohort was 21.3 [16.7 – 25.9] months and significantly longer for those receiving ARPI with 177 Lu-PSMA-617 (NR [NE – NE]) as compared to 177 Lu-PSMA-617 alone (15.3 mo [11.6 – 19.1], p < 0.001). However, on multivariate analysis including known prognostic variables, use of ARPI was not independently associated with improved survival (HR = 1.03 [CI: 0.68 to 1.55], p = 0.891). Conclusions: Patients receiving an ARPI with 177 Lu-PSMA-617 were more likely to complete all 6 cycles of treatment, but no clear differences in survival were observed on multivariate analysis.

136 Background: The ECOG performance status (PS) is a critical measure for evaluating functional capacity and determining clinical trial eligibility in patients with advanced prostate cancer. We aim to assess the capability of large language models (LLMs) to accurately infer ECOG PS from unstructured oncology notes. Methods: This retrospective study included patients with metastatic castration-resistant prostate cancer (mCRPC) receiving Lutetium-177–PSMA-617 ( 177 Lu) at the Mayo Clinic (2022-2023). Baseline ECOG PS scores were manually curated by a trained clinician from clinical notes recorded within four weeks prior to 177 Lu initiation. Structured zero-shot prompts were then used to prompt GPT-4 to infer ECOG PS scores from unstructured oncology notes. Total dataset was randomly sampled to prompt development (20%) and held-out test set (80%). Three experimental setups were employed to assess model performance: (i) utilizing unmasked clinical notes, (ii) clinical notes with only numerical ECOG PS scores masked while retaining narrative descriptive/assessment by oncologist of ECOG score, and (iii) clinical notes with both numerical scores and narrative ECOG descriptors masked. Final performance was assessed using evaluation metrics (precision, recall, F1 score) on the held-out test set. Results: A total of 240 patients (n: 50 prompt development; n: 190 test set) were included in the analysis. The median age was 70 (IQR: 66-76); a majority of patients were White (n: 229; 95%) and non-Hispanic (n: 233; 97%). The most prevalent ECOG PS score was 0 (n: 140; 58%), followed by 1 (n: 88; 37%), and 2 (n: 12; 5%). The performance was numerically higher when using unmasked notes (weighted average precision: 0.89; recall: 0.89; F1: 0.89), compared to using notes with masked scores only (weighted average precision: 0.76; recall: 0.79; F1: 0.75) and notes with both scores and narrative masked (weighted average precision: 0.57; recall: 0.53; F1: 0.48). Metrics by each ECOG PS score is outlined (Table). Conclusions: Large language models demonstrate significant potential in precisely extracting ECOG performance status from unstructured oncology notes, offering valuable support for treatment decision-making and clinical trial enrollment. However, model performance declines as reliance on implicit reporting within clinical notes increases. Future efforts focusing on prediction of ECOG using a series of notes over time will be the next step. ECOG Unmasked oncology notes ECOG Score masked only ECOG Score + ECOG narrative marked Precision Recall F1 Score Precision Recall F1 Score Precision Recall F1 Score 0 0.89 0.92 0.91 0.75 0.93 0.83 0.59 0.74 0.66 1 0.88 0.86 0.87 0.76 0.74 0.75 0.19 0.47 0.27 2 1.00 0.82 0.90 0.78 0.33 0.47 0.78 0.16 0.26

134 Background: Prostate cancer patients with visceral metastases at diagnosis are often treated using one-size-fits-all approach. For example, CHAARTED criteria treats any visceral metastasis as high volume. Therefore, we aimed to assess overall survival (OS) by site of visceral involvement in patients with metastatic PCa. Methods: Surveillance, Epidemiology, and End Results (SEER) database (2010-2021) was queried to obtain case listing data on metastatic PCa patients from 17 registries. The patients were categorized into three groups based on visceral metastasis at diagnosis: lung metastasis, liver metastasis, and others. Kaplan-Meier survival analysis was performed to estimate median OS with 95% confidence interval (CI). Log-rank test was conducted to assess if survival was significantly different among the groups. Cox proportional hazard regression analysis was performed to assess the magnitude of difference among the groups. P-value <0.05 established statistical significance. Results: This analysis included 21,007 metastatic PCa patients (lung: n=319; liver: n=123; brain: 107; bone: 20538). Metastatic PCa population with lung metastasis exhibited a median OS of 60 months (95% CI: 48-72) compared to 24 months (95% CI: 12-36) in population with liver metastases and to 36 months (95% CI: 36-36) in population with other metastases. The difference in OS among the groups was statistically significant (p < 0.001). Compared to population with lung metastasis, those with liver (HR: 1.91; 95% CI: 1.41-2.57) but not those with other metastases (1.16; 0.97-1.39) had significantly worse OS adjusting for Gleason score, PSA and race (Table). Conclusions: In metastatic PCa, patients with lung-only metastasis have significantly better overall survival and should be considered differently in prognostication to liver metastases. Consideration of site of visceral metastasis is critical for treatment intensification decisions. Limitations of this analysis include lack of accounting for confounding relationships at individual patient level. Variable Hazard Ratio (95% CI) Metastases (Lung) REFERENCE Metastases (Liver) 1.91 (1.41-2.57) Metastases (Other) 1.16 (0.97-1.39) Gleason score≥8 REFERENCE Gleason score <8 0.68 (0.64 -0.72) PSA - ng/dL 1.01 (1.00 – 1.01) Race (White) REFERENCE Race (Black) 0.99 (0.94 – 1.05) Race (Asian or Pacific Islander) 0.70 (0.64 – 0.76)

341 Background: Specialist clinicians (oncologists, urologists, radiation oncologists) read through free text reports of prostate MRI and biopsy reports to risk stratify patients during initial consultation of localized PCa. This laborious and time-consuming process is repeated for each day during the clinic. Herein, we propose a hybrid stratification framework utilizing a combination of large language model (LLM) and rule-based programming to automate this process. Methods: This retrospective study included patients with localized PCa (2004-2024) presenting to care at Mayo Clinic with at least one positive prostate biopsy and MRI report available. The state-of-the-art LLM – GPT4 – was utilized using a structured zeroshot prompt to extract relevant phenotypic variables (clinical T stage, Gleason patterns, grade group, extent of specimen/cores involved, prostate size/volume) from unstructured MRI and biopsy reports. Using a rule-based algorithm, the extracted variables and PSA at PCa diagnosis were used to categorize each patient into one of the six NCCN risk categories (very low [VL], low [L], favorable intermediate [FInt], unfavorable intermediate [UFInt], high risk (HR), very high risk [VHR]). Prompts were iteratively developed using ~5% of the total dataset and validated on ~10% of the dataset. Final performance was assessed against a held-out expert annotated test dataset using evaluation metrics (accuracy, F1-score, precision, recall). Additionally, manual annotation was conducted by two independent novice reviewers to compare machine annotated and novice-human annotated risk categorization. Results: A total of 397 patients were included in the evaluation. The median age at diagnosis was 64.8 (IQR: 59.7-68.5); majority of the men were White (n: 366; 92%) and non-Hispanic (n: 371; 94%). The most prevalent risk category was UFint (n: 151; 38%) followed by FInt (n: 80; 20.2%), HR (n: 76; 19.1%), VHR (n: 64; 16.1%), L (21: 5.3%), and VL (n: 5; 1.3%). Novice reviewers achieved an accuracy of 79%, precision of 80%, recall of 82% and a F1-score of 0.81. The hybrid stratification framework using GPT4 achieved an accuracy of 89%, precision of 89%, recall of 88% and a F1-score of 0.88. Among 43 errors by GPT4 risk stratification, majority of the errors were due to miscategorization of clinical T-stage (n: 21; 49%) followed by erroneous number of positive cores (n: 13; 30%). However, additional evaluation showed that GPT4 achieved numerically higher performance (accuracy: 90%; precision: 91%; recall: 90%; F1: 0.90) compared to novice reviewers (accuracy: 77%; precision: 89%; recall: 77%; F1: 0.82) for ascertaining clinical T-stage. Conclusions: Large language models exhibited superior performance than novice clinicians for risk stratification in patients with localized PCa. As the next step, we intend to validate our framework for automated risk classification in a prospective manner.

184 Background: We update evidence on mHSPC when new evidence becomes available (PMID: 36862387). ARANOTE trial prompted us to assess the comparative efficacy of darolutamide (DARO) with other ARPi agents in mHSPC using the living network meta-analysis (LNMA). Methods: This LNMA is conducted using the living interactive evidence (LIvE) synthesis framework. Phase III trials assessing systemic treatments in mHSPC are included. Mixed treatment comparisons were made using a frequentist network meta-analysis. P-scores were computed to assess relative treatment rankings with higher values indicating potentially better efficacy. Results: This report of LNMA includes a total 11 trials (12628 patients; 12 unique treatment options) as of October 1 st , 2024. Overall results were consistent with prior updates. Analysis limited to the ARPi trials showed that inhigh volume, abiraterone acetate (AAP)+androgen deprivation therapy (ADT) (HR: 0.46; 95% CI: 0.40-0.53; rank 1), apalutamide (APA)+ADT (0.53; 0.41-0.68; rank 3), DARO+ADT (0.60; 0.44-0.81; rank 4) and enzalutamide (E)+ADT (0.48; 0.41-0.56; rank 2) significantly improved radiographic progression-free survival (rPFS) compared to ADT alone. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (1.30; 0.94-1.82), APA+ADT (1.14; 0.77-1.67) and E+ADT (1.24; 0.89-1.79). Inlow volume, AAP+ADT (0.48; 0.37-0.63; rank 4), APA+ADT (0.36; 0.22-0.58; rank 3), DARO+ADT (0.30; 0.15-0.60; rank 1) and E+ADT (0.32; 0.25-0.41; rank 2) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.67; 0.47-0.96) in low volume. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (0.62; 0.30-1.30), APA+ADT (0.83; 0.36-1.92) and E+ADT (0.93; 0.45-1.94). In synchronous disease, AAP+ADT (0.58; 0.51-0.67; rank 4), APA+ADT (0.49; 0.39-0.62; rank 2), DARO+ADT (0.59; 0.44-0.79; rank 3) and E+ADT (0.42; 0.36-0.50; rank 1) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.73; 0.59-0.89) in synchronous disease. In metachronous disease, DARO+ADT (0.34; 0.17-0.67; rank 1), APA+ADT (0.41; 0.22-0.77; rank 2) and E+ADT (0.44; 0.35-0.57; rank 3) significantly improved rPFS compared to ADT. No statistically significant differences were observed with DARO+ADT compared to APA+ADT (0.83; 0.33-2.08) and E+ADT (0.76; 0.37-1.57) in metachronous disease. The results were consistent for overall survival. Conclusions: Current evidence suggests no significant differences for darolutamide as compared to other ARPi agents. Enzalutamide may be preferred over AAP in low-volume or synchronous disease. Given similar efficacy, choice of ARPi requires careful consideration of patient-specific factors including cost, accessibility and toxicity.

866 Background: Emerging data suggests that Language Models (LLMs) provide an unprecedented opportunity to extract critical data from the unstructured pathology reports. However, the performance of state of art (SOTA) models on clinically meaningful tasks such as assessment of pathologic responses in patients receiving neoadjuvant chemotherapy (NAC) in muscle invasive bladder cancer(MIBC) is not well known. Methods: This retrospective cohort included patients with pathologically confirmed Muscle-Invasive Bladder Cancer from the year 2018 to 2024. Selected patients included those who underwent Neoadjuvant chemotherapy followed by definitive cystectomy at Mayo Clinic. Gold standard labels were manually curated by trained clinicians. The state of the art (SOTA) LLM – GPT4 – was utilized using a structured zero-shot prompt to extract relevant phenotypic variables such as tumor site, tumor histology and additional histological variants if any, presence of in-situ / invasive carcinoma, presence of lamina propria / muscularis propria / lympho-vascular / perineural invasion, and T-stage from unstructured pathology reports. Structured prompts were iteratively developed for each data variable and validated using ~5% of the total dataset. The extracted variables were manually compared and labelled as True positive / True negative / False positive / False negative. Final performance was assessed against a held-out expert annotated test dataset using evaluation metric (accuracy). Results: A total of 200 reports from 99 patients were extracted by the LLM. Of the 200 reports, 41 duplicate / other pathology reports (e.g. – autopsy, cholecystectomy, etc.) were removed during final assessment. Significant characteristics such as tumor site, histology, and additional histology variant had an accuracy of 89%, 87%, and 91% respectively. Notably, accuracy of detection of in-situ versus invasive carcinoma was 70% versus 91%. Specific variables such as lamina propria / muscularis propria invasion / T-stage / lympho-vascular invasion / perineural invasion were answered with the accuracy of 75%, 75%, 67%, 90%, and 94% respectively. Conclusions: LLMs have the potential to extract critical data from unstructured pathology reports and automate assessment of pathologic responses in MIBC patients receiving NAC. Iterative improvements in performance can be achieved by improving prompt design using expert clinical input. This process can be potentially scaled to automate cancer registries.

Abstract Objective Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world 2-reviewer process. Materials and Methods A dataset of 10 trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and held-out test sets (n = 17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the 2 LLMs were considered concordant if they were the same for a given variable. The discordant responses from each LLM were provided to the other LLM for cross-critique. Accuracy, ie, the total number of correct responses divided by the total number of responses, was computed to assess performance. Results In the prompt development set, 110 (96%) responses were concordant, achieving an accuracy of 0.99 against the gold standard. In the test set, 342 (87%) responses were concordant. The accuracy of the concordant responses was 0.94. The accuracy of the discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. Of the 49 discordant responses, 25 (51%) became concordant after cross-critique, increasing accuracy to 0.76. Discussion Concordant responses by the LLMs are likely to be accurate. In instances of discordant responses, cross-critique can further increase the accuracy. Conclusion Large language models, when simulated in a collaborative, 2-reviewer workflow, can extract data with reasonable performance, enabling truly “living” systematic reviews.

Introduction Symptom burden in myeloproliferative neoplasms (MPN) is quantified using total symptom scores (TSS) calculated from the patient-reported scores for clinically meaningful symptoms. The impact of demographic, clinical, and laboratory features on these symptoms remains unclear and warrants further investigation. Methods Patients with polycythemia vera (PV), essential thrombocythemia (ET), and myelofibrosis (MF) were identified from the retrospective chart review. TSS, scores of individual symptoms (fatigue, early satiety, abdominal discomfort, inactivity, concentration difficulties, fever, night sweats, itching, bone pain, weight loss), demographic characteristics (race, ethnicity, age, and gender), clinical features (time since diagnosis, depression status, obesity status), laboratory results and season at the time of visit when the first TSS assessment was performed were recorded. Multicollinearity and homoscedasticity were assessed using variance inflation factors and visual inspection of the data, respectively. A multivariable regression analysis was performed using a robust variance estimator. A p-value &lt;0.05 indicated a statistically significant association with the symptom scores. Results The chart review identified 557 (PV: 201; ET: 191; MF: 165) patients. The median age of patients with PV, ET, and MF was 65 (range: 27-93), 65 (20-96), and 69 (27-96) years, respectively. Most patients were white (PV: 92%; ET: 90%; MF: 93%) and non-Hispanic or Latino (PV: 93%; ET: 96%; MF: 92%). Most patients with PV and ET were female (PV: 51%; ET: 73%), while most patients with MF were male (55%). Among the included patients 22% (PV: 21%; ET: 27%; MF: 18%) had the diagnosis of depression at the time of the clinical visit. TSS was highest in MF (mean (SD): 18.1±14.6), followed by PV (15.7±14.5) and ET (15.1±14.0) patients. Fatigue was the most reported symptom by the patients (PV: 81%; ET: 82%; MF: 84%), whereas weight loss was the least reported symptom (PV: 4.5%; ET: 6.3%; MF: 16%). With regards to the TSS multivariable regression analysis showed that age (beta coefficient: 0.2; p=0.01), depression (7.2; 0.008), and low hemoglobin (1.2; 0.003) in PV, depression (11.7; 0.001) in ET, and depression (9.5; 0.001), splenomegaly (6.9; 0.01), and WBC count (0.16; 0.048) in MF patients were significantly associated with TSS. In terms of individual symptoms in PV, depression was associated with fatigue (beta coefficient: 1.31; p-value: 0.01) and concentration difficulties (1.7; &lt;0.001), high WBC count was associated with early satiety (0.09; 0.03), and non-white race was associated with fever (1.4; 0.007). In ET, depression was associated with fatigue (1.6; &lt;0.001), abdominal discomfort (1.4; &lt;0.001), inactivity (2.5; &lt;0.001), concentration difficulties (2.4; &lt;0.001) and night sweats (1.4; &lt;0.001), anemia was associated with fatigue (1.2; 0.03), fewer months since diagnosis was associated with inactivity (0.01; 0.02) and young age was associated with bone pain (0.02; 0.03). In MF, depression was associated with fatigue (1.8; 0.001), concentration difficulties (1.3; 0.01), fever (1.1; 0.02), and night sweats (2.2; 0.03), low hemoglobin was associated with fatigue (0.46; &lt;0,001) and early satiety (0.31; 0.003), high WBC count was associated with fatigue (0.03; 0.04) and early satiety (0.04; 0.02), low platelet count was associated with abdominal discomfort (0.002; 0.03), splenomegaly was associated with inactivity (2.27; 0.003) and female gender was associated with itching (0.57; 0.01). Conclusions This study demonstrates that depression in PV and ET and depression, low hemoglobin, and a high WBC count in MF contribute to multiple symptoms seen in these patients. Identifying demographic groups at risk of a higher symptom burden and addressing the comorbidities that drive symptoms, in addition to disease-directed therapy, is crucial to improving the quality of life in these patients.