Khakwani, Kaneez Zahra Rubab

5108 Background: We previously proposed a hybrid framework combining LLMs and rule based algorithm (RBA) for automating risk stratification in locPCa. Herein, we aim to validate the risk stratification agent (RSA) in a prospective cohort, develop & evaluate the treatment recommendation agent (TRA) based on NCCN guidelines, and develop an interactive interface to facilitate clinicians for accurate risk stratification and treatment recommendations at the point of care. Methods: This study included pts with locPCa (2004-2024) presenting at Mayo Clinic with at least 1 positive prostate biopsy and MRI report available. For RSA prospective validation, GPT4 extracted key phenotypic variables (PSA, T stage, prostate volume, number of cores, Gleason patterns, grade group) from unstructured MRI and biopsy reports using a zeroshot prompt. An RBA then classified pts into NCCN risk groups. The agent performance was compared with the treating clinician's documentation and evaluated against gold-standard labels manually annotated by two independent clinicians. For development of TRA, two experiments were performed using GPT4 with and without retrieval-augmented generation (RAG) to generate treatment plans. Generated treatment plans were evaluated using NCCN guideline-informed treatment decision tree based algorithm (DTA). Evaluation metrics, weighted – accuracy (acc) and F1, were computed. A clinician facing interface (lisr.org/risk) was developed to provide accurate risk stratification and treatment plans. Results: A total of 858 pts were included (500 for prospective validation, 358 for treatment recommendations). Prospective validation for RSA demonstrated a higher F1 score of 0.89 compared to the treating clinician (F1: 0.58). Treatment recommendation experiments showed that GPT4 with RAG achieved higher acc (64% full – all correct treatment options and 36% partial – at least 1 correct treatment option) compared to GPT4 alone (35% and 65%, respectively). Sensitivity analysis using DTA-informed GPT4 note generation achieved 94% full acc and 6% partial acc. GPT4 alone generated hallucinated treatment options in 71% of cases, while GPT4 with RAG reduced this to 32%. Conclusions: The multi-agent framework based on LLM and RBA achieves high accuracy in risk stratification and treatment recommendation for locPCa. A multi-agent framework with an interactive interface holds high promise to enable efficient, accurate decision-making, and improve locPCa management at the point of care Performance evaluation. RSA(acc; F1) % TRA(full acc; partial acc) % Clinician RSA* GPT4 RAG+GPT4 DTA+GPT4 Overall 60; 58 89; 89 29; 71 64; 36 94; 6 V. High 94; 33 96; 96 55; 45 69; 31 93; 7 High 46; 52 92; 89 - - - Int. Unf 80; 61 91; 90 5; 95 38; 62 93; 7 Int. Fav 57; 64 84; 87 0; 100 98; 2 100; 0 Low 57; 65 87; 80 100; 0 95; 5 90; 10 V. Low 8; 13 50; 67^ - - - *Statistical significance (p<0.01) except for ^V. Low.

ABSTRACTBackgroundThe treatment landscape for multiple myeloma continues to evolve. Recently, the addition of anti‐CD38 monoclonal antibodies (mAbs) to the triplet regimen, comprising a proteasome inhibitor, an immunomodulatory agent, and a steroid, for transplant‐eligible newly diagnosed multiple myeloma (TENDMM) has shown promising results.AimsTo evaluate the overall efficacy and safety of quadruplet therapy with an anti‐CD38 mAb compared to a triplet regimen.MethodsA systematic search of Medline, Scopus, and EMBASE databases from inception to July 2024 identified relevant randomized controlled trials (RCTs). Efficacy and safety outcomes were derived using random‐effects meta‐analysis. Summarized outcomes include hazard ratios (HR) for progression‐free survival (PFS) and overall survival (OS), odds ratios (OR) for response rates, measurable residual disease (MRD) negativity rate, and grade 3 or higher adverse events (G ≥ 3 AEs).ResultsFive RCTs involving 2963 patients were included. A statistically significant PFS was observed for quadruplet therapy when compared to the triplet regimen (HR 0.44; 95% Confidence Interval [CI] 0.35–0.56). PFS benefit was consistent for the standard risk (SR) group (HR 0.38; 95% CI 0.27–0.52) and high risk (HiR) group (HR 0.62; 95% CI 0.41–0.92). No statistically significant benefit was observed for OS (HR 0.55; 95% CI 0.28–1.08). A statistically significant benefit was observed for the overall response rate (OR 1.77; 95% CI 1.02–3.06) and MRD negativity rate (OR 2.67; 95% CI 1.79–3.99). No significant differences were observed for G ≥ 3 AE (OR 1.21; 95% CI 0.92–1.58), lymphopenia (OR 1.09; 95% CI 0.62–1.89), and anemia (OR 1.06; 95% CI 0.83–1.37). However, a significantly increased risk was observed for all‐grade thrombocytopenia (OR 1.64; 95% CI 1.37–1.97), neutropenia (OR 2.24; 95% CI 1.67–3.02) and infections (OR 1.88; 95% CI 1.07–3.31).ConclusionQuadruplet therapy demonstrated a favorable efficacy and safety profile, with consistent benefit across subgroups. The findings support its potential as the new standard of care for TENDMM.

270 Background: Inconsistent reporting of subgroup data across clinical trials makes interpretation and application of the evidence from trials challenging. We conducted a living systematic review to summarize evidence for American Society of Clinical Oncology (ASCO) guidelines for management of metastatic castration resistant prostate cancer (mCRPC). Here, we provide an overview of reporting patterns of subgroup data across mCRPC trials. Methods: MEDLINE and EMBASE were systematically searched from each database's inception through September 20 th , 2024, to identify phase II/III/IV randomized clinical trials assessing the efficacy and safety of treatment options in mCRPC. For outcomes of overall survival (OS) and progression free survival (PFS), data from the subgroup analyses was collected and compared across trials to assess heterogeneity in the reporting of subgroups. Results: This analysis included 141 trials (phase II: 83; phase III: 56; phase IV: 2). OS was reported by 113 (80%) trials; 32 (28%) reported OS by any subgroups with 95 unique subgroup categories. PFS was reported by 110 (78%) trials; 23 (21%) reported PFS by any subgroups with 63 unique subgroup categories. Even among consistently reported subgroups, such as age, ECOG performance status, race, geographical area, and Gleason score, considerable heterogeneity in different subgroup levels/thresholds was observed across trials. Top ten most commonly reported subgroups are summarized (Table). Conclusions: Lack of consistent reporting for subgroup analysis in mCRPC trials preclude meaningful conclusions about clinically relevant subgroup and synthesizing evidence for clinical guidelines. A standardized framework for consistent reporting of subgroup data across trials can help inform clinical practice and clinical practice guidelines. OS PFS Number of trials 113 110 Reporting subgroup analysis – N (%) 32 (28) 23 (21) Commonly reported subgroups – (N) Age (26)ECOG-PS (26)LDH (21)PSA (18)Geographic area (15)Alkaline phosphatase (14)Visceral disease (14)Gleason score (9)Race (9)Brief pain inventory level (8) Age (21) ECOG-PS (19)PSA (18)LDH (14)Geographic area (13)Visceral disease (11)Gleason score (10)Alkaline phosphatase (9)Bone metastasis (9)Disease location (9) Age – (N) <65 (14); ≥65 (9); ≤68 (1); ≥69 (1); <70 (5); ≥70 (5); ≤74 (4); 65-75 (5); ≤ 71 (1); >71 (1); ≥75 (11); >Median (1); <Median (1) <65 (12); 65-74 (6); ≥65 (6); <70 (5); ≥70 (5); <75 (4); ≥75 (10); 76-80 (1); >80 (1) ECOG-PS – (N) 0 (15); 0-1 (11), 1 (11); 1-2 (5); 2 (12) 0 (15); 0-1 (5), 1 (12); 1-2 (2); 2 (6) Race – (N) White (9)Non-white (6)African American/black (2)Asian (2) White (8)Non-white (5)African American/black (2)Asian (4) Geographic Area – (N) Europe (8)North America (14)United States (1)Australia (1)Asia (2) Europe (11)North America (13)United States (1)Australia (1)Asia (2)North and South America (2) Gleason Score – (N) 2-6 (1); 7 (1); 8 (1); <8 (8); ≥ 8 (8); 9-10 (1) <8 (10); ≥ 8 (10)

341 Background: Specialist clinicians (oncologists, urologists, radiation oncologists) read through free text reports of prostate MRI and biopsy reports to risk stratify patients during initial consultation of localized PCa. This laborious and time-consuming process is repeated for each day during the clinic. Herein, we propose a hybrid stratification framework utilizing a combination of large language model (LLM) and rule-based programming to automate this process. Methods: This retrospective study included patients with localized PCa (2004-2024) presenting to care at Mayo Clinic with at least one positive prostate biopsy and MRI report available. The state-of-the-art LLM – GPT4 – was utilized using a structured zeroshot prompt to extract relevant phenotypic variables (clinical T stage, Gleason patterns, grade group, extent of specimen/cores involved, prostate size/volume) from unstructured MRI and biopsy reports. Using a rule-based algorithm, the extracted variables and PSA at PCa diagnosis were used to categorize each patient into one of the six NCCN risk categories (very low [VL], low [L], favorable intermediate [FInt], unfavorable intermediate [UFInt], high risk (HR), very high risk [VHR]). Prompts were iteratively developed using ~5% of the total dataset and validated on ~10% of the dataset. Final performance was assessed against a held-out expert annotated test dataset using evaluation metrics (accuracy, F1-score, precision, recall). Additionally, manual annotation was conducted by two independent novice reviewers to compare machine annotated and novice-human annotated risk categorization. Results: A total of 397 patients were included in the evaluation. The median age at diagnosis was 64.8 (IQR: 59.7-68.5); majority of the men were White (n: 366; 92%) and non-Hispanic (n: 371; 94%). The most prevalent risk category was UFint (n: 151; 38%) followed by FInt (n: 80; 20.2%), HR (n: 76; 19.1%), VHR (n: 64; 16.1%), L (21: 5.3%), and VL (n: 5; 1.3%). Novice reviewers achieved an accuracy of 79%, precision of 80%, recall of 82% and a F1-score of 0.81. The hybrid stratification framework using GPT4 achieved an accuracy of 89%, precision of 89%, recall of 88% and a F1-score of 0.88. Among 43 errors by GPT4 risk stratification, majority of the errors were due to miscategorization of clinical T-stage (n: 21; 49%) followed by erroneous number of positive cores (n: 13; 30%). However, additional evaluation showed that GPT4 achieved numerically higher performance (accuracy: 90%; precision: 91%; recall: 90%; F1: 0.90) compared to novice reviewers (accuracy: 77%; precision: 89%; recall: 77%; F1: 0.82) for ascertaining clinical T-stage. Conclusions: Large language models exhibited superior performance than novice clinicians for risk stratification in patients with localized PCa. As the next step, we intend to validate our framework for automated risk classification in a prospective manner.

184 Background: We update evidence on mHSPC when new evidence becomes available (PMID: 36862387). ARANOTE trial prompted us to assess the comparative efficacy of darolutamide (DARO) with other ARPi agents in mHSPC using the living network meta-analysis (LNMA). Methods: This LNMA is conducted using the living interactive evidence (LIvE) synthesis framework. Phase III trials assessing systemic treatments in mHSPC are included. Mixed treatment comparisons were made using a frequentist network meta-analysis. P-scores were computed to assess relative treatment rankings with higher values indicating potentially better efficacy. Results: This report of LNMA includes a total 11 trials (12628 patients; 12 unique treatment options) as of October 1 st , 2024. Overall results were consistent with prior updates. Analysis limited to the ARPi trials showed that inhigh volume, abiraterone acetate (AAP)+androgen deprivation therapy (ADT) (HR: 0.46; 95% CI: 0.40-0.53; rank 1), apalutamide (APA)+ADT (0.53; 0.41-0.68; rank 3), DARO+ADT (0.60; 0.44-0.81; rank 4) and enzalutamide (E)+ADT (0.48; 0.41-0.56; rank 2) significantly improved radiographic progression-free survival (rPFS) compared to ADT alone. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (1.30; 0.94-1.82), APA+ADT (1.14; 0.77-1.67) and E+ADT (1.24; 0.89-1.79). Inlow volume, AAP+ADT (0.48; 0.37-0.63; rank 4), APA+ADT (0.36; 0.22-0.58; rank 3), DARO+ADT (0.30; 0.15-0.60; rank 1) and E+ADT (0.32; 0.25-0.41; rank 2) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.67; 0.47-0.96) in low volume. No statistically significant differences were observed with DARO+ADT compared to AAP+ADT (0.62; 0.30-1.30), APA+ADT (0.83; 0.36-1.92) and E+ADT (0.93; 0.45-1.94). In synchronous disease, AAP+ADT (0.58; 0.51-0.67; rank 4), APA+ADT (0.49; 0.39-0.62; rank 2), DARO+ADT (0.59; 0.44-0.79; rank 3) and E+ADT (0.42; 0.36-0.50; rank 1) significantly improved rPFS compared to ADT. E+ADT significantly improved rPFS compared to AAP+ADT (0.73; 0.59-0.89) in synchronous disease. In metachronous disease, DARO+ADT (0.34; 0.17-0.67; rank 1), APA+ADT (0.41; 0.22-0.77; rank 2) and E+ADT (0.44; 0.35-0.57; rank 3) significantly improved rPFS compared to ADT. No statistically significant differences were observed with DARO+ADT compared to APA+ADT (0.83; 0.33-2.08) and E+ADT (0.76; 0.37-1.57) in metachronous disease. The results were consistent for overall survival. Conclusions: Current evidence suggests no significant differences for darolutamide as compared to other ARPi agents. Enzalutamide may be preferred over AAP in low-volume or synchronous disease. Given similar efficacy, choice of ARPi requires careful consideration of patient-specific factors including cost, accessibility and toxicity.

Abstract Objective Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world 2-reviewer process. Materials and Methods A dataset of 10 trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and held-out test sets (n = 17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the 2 LLMs were considered concordant if they were the same for a given variable. The discordant responses from each LLM were provided to the other LLM for cross-critique. Accuracy, ie, the total number of correct responses divided by the total number of responses, was computed to assess performance. Results In the prompt development set, 110 (96%) responses were concordant, achieving an accuracy of 0.99 against the gold standard. In the test set, 342 (87%) responses were concordant. The accuracy of the concordant responses was 0.94. The accuracy of the discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. Of the 49 discordant responses, 25 (51%) became concordant after cross-critique, increasing accuracy to 0.76. Discussion Concordant responses by the LLMs are likely to be accurate. In instances of discordant responses, cross-critique can further increase the accuracy. Conclusion Large language models, when simulated in a collaborative, 2-reviewer workflow, can extract data with reasonable performance, enabling truly “living” systematic reviews.

4591 Background: EV-302 and CheckMate 901 demonstrated significant survival benefit with enfortumab vedotin-pembrolizumab (EVPembro) and gemcitabine-cisplatin-nivolumab (GemCisNivo) as 1L therapy, respectively, in patients (pts) with la/mUC. However, whether these treatments are cost-effective or not remains unclear. Methods: A three-state Markov model (progression-free, progression and death) was developed. State utilities were derived from published literature. State transition probabilities were informed from point-probabilities and hazard ratios for OS and PFS obtained from the latest follow-up of eligible trials. Average sales price (2024 USD) for individual treatments (excluding ancillary charges) were obtained from the center of medicare and medicaid services (US payer’s perspective). Treatment strategies were modeled in accordance with the dose/schedule reported in the eligible trials. Pts who received GemCis or Gem-carboplatin (GemCarbo) and did not progress were modeled to receive Avelumab maintenance. Half-cycle corrected costs and utilities were accrued over a 10-year lifetime horizon and were discounted at 3%. Monte Carlo simulation was used to estimate quality adjusted life years (QALYs) and incremental cost-effectiveness ratios (ICERs). A willingness-to-pay threshold (WTP) of 100,000$ per QALY was used. Results: In cisplatin-eligible pts, GemCisNivo demonstrated a 0.8 QALY gain over GemCis at an ICER of 21,127 ($/QALY). GemCisNivo was also associated with an incremental net monetary benefit (INMB) of 63,099$ relative to GemCis. EVPembro demonstrated a 2.4 QALY gain over GemCis at an ICER of 915,283 and a 1.6 QALY gain over GemCisNivo at an ICER of 1,406,308. In cisplatin-ineligible pts, EVPembro demonstrated a 2.4 QALY gain over Gem-carboplatin (GemCarbo) at an ICER of 934,761. Price threshold analyses demonstrated that EVPembro is likely to be cost-effective at a cost of 12,344$ for cisplatin-eligible and 13,374$ per cycle for cisplatin-ineligible pts (original cost: 35,665.60$ per cycle). Sensitivity analysis using a total of 12 cycles for EVPembro showed cost-effectiveness at an ICER of 69,594 relative to GemCisNivo in cisplatin-eligible pts, and at an ICER of 53,099 relative to GemCarbo in cisplatin-ineligible pts. Conclusions: In terms of US payer’s perspective, GemCisNivo is most likely to be cost-effective in cisplatin-eligible pts at the WTP of 100,000 $/QALY. Limiting the number of cycles for EVPembro may render it as a cost-effective 1L treatment in la/mUC pts regardless of cisplatin-eligibility status.

e19534 Background: Immunomodulatory drugs, proteasome inhibitors plus dexamethasone are the standard triplet induction regimen for TENDMM. We analyzed the efficacy of adding either anti-CD38 monoclonal antibodies (mAbs) or anti-SLAMF7 antibody to the triplet therapy. Methods: Ovid MEDLINE, EMBASE and published abstracts were systematically searched from each database’s inception through 01-31-2024 to identify II/III trials assessing quadruplet therapy versus triplet therapy in TENDMM patients. Hazard ratios (HR) for progression-free survival (PFS) and overall survival (OS), odds ratios (OR) for overall response rates (ORR), measurable residual disease (MRD) negativity rate (with sensitivity of 10 -5) and grade 3 or higher treatment-related adverse events (G≥3 TRAE) were summarized using random effects meta-analysis via inverse variance approach. Additional subgroup analyses based on the cytogenetic risk were performed. P-value of interaction (Pint ) < 0.1 was pre-specified as statistically significant for the difference between the two groups. Results: 3806 studies were screened and seven trials were included with a total of 3546 patients. Four trials (2303 patients) assessed the addition of anti-CD-38 mAbs and three (1243 patients) assessed anti-SLAMF7 mAb, elotuzumab to the triplet regimen. Quadruplet therapy significantly improved PFS as compared to triplet (HR: 0.63; 95% CI: 0.43-0.92), and the PFS benefit was also observed in standard risk group (HR 0.5; 95% CI 0.2-0.9); however, results were insignificant for high-risk group (HR 0.8; 95% CI 0.62-1.93). No significant OS benefit was observed (HR: 0.79; 95% CI 0.49–1.26); however, data for three trials (CASSIOPEIA, DSMM XVII, PERSEUS) is immature, with longer-term follow-up ongoing. Quadruplet therapy showed significant results for very good partial response or better (OR: 1.7; 95% CI 1.2-2.36), stringent complete response (OR: 1.74; 95% CI 1.27-2.07) and complete response or better (OR: 1.91; 95% CI 1.19-3.06). MRD rate shows benefit with quadruplet therapy (OR: 2.59; 95% CI 2.22-3.01). Improved MRD rate was also observed for high-risk (OR 2.13; 95% CI 1.4-3.2) and standard-risk group (OR 2.58; 95% CI 1.68-3.94). Secondary analysis for efficacy of anti-CD38 mAbs showed benefit for PFS (HR: 0.44; 95% CI: 0.35-0.56), ORR (OR: 1.77;95% CI: 1.02-3.06), and MRD (OR: 2.98; 95% CI 1.8-4.8). G ≥3 TRAE did not significantly differ between the groups (OR: 1.05; 95% CI 0.97-1.12). However, quadruplet therapy showed significant risk of infection (OR: 1.2; 95% 1.05-1.35) and thrombocytopenia (OR: 1.41; 95% CI 1.24-1.59). Conclusions: Quadruplet therapy improved PFS, ORR, and MRD rates compared to triplet regimen and has demonstrated promising benefits to be considered the new standard of care for TENDMM patients.

e17059 Background: It is unclear if patients with different HRR gene-mutated mCRPC derive consistent benefit from treatment with PARPi and API combination. Approval labels for PARPi+API are inconsistent across regulatory bodies in the US (Food and Drug Administration, FDA) and Europe (European Medical Agency, EMA), adding to confusion among practicing clinicians. Therefore, we aimed to assess the differential efficacy of PARP+API therapy by different genes using up-to-date trial evidence. Methods: We maintain a living meta-analysis on mCRPC treatments that is updated when new data becomes available. Here, we report data from phase III randomized control trials (RCTs) assessing PARPi+API in 1L mCRPC. The outcomes of interest included radiographic progression-free survival (rPFS) and overall survival (OS). Hazard ratios (HR) for rPFS and OS by different genes were summarized using random effects meta-analysis via inverse variance approach. Results: We have screened 27376 references since the inception of this living review. Here, we report data from three trials (15 references) with a total of 1618 patients. Frequently reported HRR gene mutations in the included trials were BRCA2 (35%), ATM (21%), CDK12 (12.2%), CHEK2 (12.2%), BRCA1 (4.6%), and PALB2 (3.4%). In terms of rPFS, statistical benefit was observed in BRCA (0.28; 95% CI: 0.13-0.59) and CDK12 (0.58; 95% CI: 0.35-0.95) subgroups, whereas no statistically significant benefit was seen in PALB2 (0.53; 95% CI: 0.21-1.32), ATM (0.93; 95% CI, 0.57-1.53), and CHEK2 (0.92; 95% CI: 0.53-1.61) subgroups. For OS, benefit was observed in BRCA (0.47; 95%CI: 0.31-0.71) subgroup. While not statistically significant, a signal of meaningful OS benefit was observed in PALB2 (0.33; 95%CI: 0.10-1.16) but no OS benefit was observed in ATM (0.97; 95% CI: 0.57-1.67), CDK12 (0.80; 95% CI: 0.36-1.78), and CHEK2 (0.81; 95% CI: 0.37-1.75) subgroups. Conclusions: Current results show that mCRPC patients with BRCA and CDK12 alterations had delayed disease progression with PARPi+API as 1L therapy. BRCA mutations are also associated with improved OS, while patients with CHEK2 and ATM derived no significant benefit with PARPi+API combination. [Table: see text]