Rumble, Bryan

e17004 Background: Living clinical practice guidelines offer the opportunity to keep recommendations aligned with the latest evidence, but they also present a fundamental challenge: as evidence accumulates, panelists cannot sustainably query and synthesize a growing body of literature. In developing ASCO's living metastatic castration-resistant prostate cancer (mCRPC) guideline, we encountered this challenge firsthand. To address it, we developed and evaluated an interactive agentic framework that enables guideline panelists to query evidence in natural language and retrieve structured facts, tables, and figures—with the goal of extending this tool as a clinician-facing companion to enhance guideline accessibility. Methods: The mCRPC systematic review dataset supporting ASCO's clinical practice guideline was used, comprising 212 structured columns capturing trial characteristics, patient populations, and outcomes extracted from 188 references across 88 clinical trials. A large language model (LLM)-based agentic framework was developed employing a coding agent that translates natural language queries into executable scripts for structured evidence retrieval. Claude (Haiku 4.5, Sonnet 4.5, and Opus 4.5) served as the core LLM agents. We constructed a benchmark of 50 fact-based queries with gold-standard answers targeting trial identification, treatment modalities, and endpoint characteristics (e.g., "Which trials evaluated systemic monotherapy?"). Performance was evaluated using exact-match accuracy for final answers, and precision, recall, and F1 score for trial-level retrieval. Results: Across 50 queries, exact-match accuracy ranged from 82% to 86%. Haiku 4.5 and Sonnet 4.5 each achieved 82% (42/50), while Opus 4.5 achieved 86% (43/50). Precision ranged from 0.93–0.97, recall from 0.94–0.96, and F1 scores from 0.91–0.94. Conclusions: This agentic framework enables scalable, auditable, natural language interrogation of guideline evidence, addressing a critical bottleneck in living guideline maintenance. For panelists, it supports rapid, reproducible evidence surveillance during recommendation development. For clinicians, it offers a foundation for an interactive companion tool that transforms static guidelines into queryable resources—enabling personalized evidence retrieval at the point of care. LLM EM Precision Recall F1 claude-haiku-4-5 0.82(42/50) 0.95 0.94 0.91 claude-sonnet-4-5 0.82(42/50) 0.97 0.96 0.93 claude-opus-4-5 0.86(43/50) 0.93 0.96 0.94

270 Background: Inconsistent reporting of subgroup data across clinical trials makes interpretation and application of the evidence from trials challenging. We conducted a living systematic review to summarize evidence for American Society of Clinical Oncology (ASCO) guidelines for management of metastatic castration resistant prostate cancer (mCRPC). Here, we provide an overview of reporting patterns of subgroup data across mCRPC trials. Methods: MEDLINE and EMBASE were systematically searched from each database's inception through September 20 th , 2024, to identify phase II/III/IV randomized clinical trials assessing the efficacy and safety of treatment options in mCRPC. For outcomes of overall survival (OS) and progression free survival (PFS), data from the subgroup analyses was collected and compared across trials to assess heterogeneity in the reporting of subgroups. Results: This analysis included 141 trials (phase II: 83; phase III: 56; phase IV: 2). OS was reported by 113 (80%) trials; 32 (28%) reported OS by any subgroups with 95 unique subgroup categories. PFS was reported by 110 (78%) trials; 23 (21%) reported PFS by any subgroups with 63 unique subgroup categories. Even among consistently reported subgroups, such as age, ECOG performance status, race, geographical area, and Gleason score, considerable heterogeneity in different subgroup levels/thresholds was observed across trials. Top ten most commonly reported subgroups are summarized (Table). Conclusions: Lack of consistent reporting for subgroup analysis in mCRPC trials preclude meaningful conclusions about clinically relevant subgroup and synthesizing evidence for clinical guidelines. A standardized framework for consistent reporting of subgroup data across trials can help inform clinical practice and clinical practice guidelines. OS PFS Number of trials 113 110 Reporting subgroup analysis – N (%) 32 (28) 23 (21) Commonly reported subgroups – (N) Age (26)ECOG-PS (26)LDH (21)PSA (18)Geographic area (15)Alkaline phosphatase (14)Visceral disease (14)Gleason score (9)Race (9)Brief pain inventory level (8) Age (21) ECOG-PS (19)PSA (18)LDH (14)Geographic area (13)Visceral disease (11)Gleason score (10)Alkaline phosphatase (9)Bone metastasis (9)Disease location (9) Age – (N) <65 (14); ≥65 (9); ≤68 (1); ≥69 (1); <70 (5); ≥70 (5); ≤74 (4); 65-75 (5); ≤ 71 (1); >71 (1); ≥75 (11); >Median (1); <Median (1) <65 (12); 65-74 (6); ≥65 (6); <70 (5); ≥70 (5); <75 (4); ≥75 (10); 76-80 (1); >80 (1) ECOG-PS – (N) 0 (15); 0-1 (11), 1 (11); 1-2 (5); 2 (12) 0 (15); 0-1 (5), 1 (12); 1-2 (2); 2 (6) Race – (N) White (9)Non-white (6)African American/black (2)Asian (2) White (8)Non-white (5)African American/black (2)Asian (4) Geographic Area – (N) Europe (8)North America (14)United States (1)Australia (1)Asia (2) Europe (11)North America (13)United States (1)Australia (1)Asia (2)North and South America (2) Gleason Score – (N) 2-6 (1); 7 (1); 8 (1); <8 (8); ≥ 8 (8); 9-10 (1) <8 (10); ≥ 8 (10)