Xie, En

Abstract Minimal residual disease (MRD) has been endorsed by the FDA Oncology Drugs Advisory Committee as an endpoint for accelerated approval in multiple myeloma based on individual patient data collected from randomized trials. However, emerging data from recent trials were not included. A novel artificial intelligence (AI)–assisted framework is proposed, which automates information identification and extraction, providing up-to-date analyses that confirm moderate trial-level and strong individual patient–level associations between MRD− rates at suspected complete response (MRD-CR) and survival endpoints in multiple myeloma. Specifically, this study utilized an AI-assisted framework that identifies relevant studies and filters critical information to analyze published data via 2 independent objectives. First, we examined the trial-level association using the coefficients of determination (R2) and its 95% confidence intervals (CI) based on published statistics of treatment effects on MRD and various endpoints. Next, we generated synthetic individual patient data with covariates through AI-curated tools to estimate the individual-level association. The AI tool searched for eligible randomized clinical trials (RCT). A total of 20 two-arm comparisons from 19 RCTs were analyzed. Trial-level analysis showed an R2 of 0.71 (95% CI, 0.52–0.89) pooling disease subpopulations. Furthermore, AI techniques were applied to create synthetic individual data, combining information extracted from Kaplan–Meier curves and subgroup analyses from published literatures. Using generated synthetic data, we estimated the individual-level correlation between MRD-CR rates and progression-free survival outcomes with a bivariate copula model and calculated a global odds ratio of 7.28 (95% CI, 5.60–8.95). Significance: (i) A novel AI-assisted framework is proposed, which automates information identification and extraction, providing rapid up-to-date analyses. (ii) Moderate trial-level and strong individual patient–level associations between MRD and various clinical endpoints in multiple myeloma are confirmed.

e19554 Background: Adding daratumumab to standard multiple myeloma (MM) regimens improves response rates and clinical outcomes. Systematic reviews (meta-analyses) allow further evaluation of its effectiveness in MM patients with high-risk cytogenetic factors but are limited by model assumptions. his study addresses these limitations using a novel synthetic individual patient data (SynIPD) generation algorithm to enable a more robust analysis. Methods: Advances in AI have enabled the creation of synthetic data that mimics the statistical properties of real patient data, empowering aggregated analyses of clinical trials. This study focuses on newly diagnosed MM (NDMM) and relapsed/refractory MM (RRMM) patients with high-risk cytogenetic profiles. Six studies were included: ALCYONE, CASSIOPEIA, and MAIA for NDMM, and CANDOR, CASTOR, and POLLUX for RRMM. Large language models extracted clinical evidence, and SynIPD was generated, incorporating cytogenetic profiles. Random-effects meta-analyses were conducted, and results were compared with pooled analyses using synthetic data. Results: Synthetic data were generated for three studies for NDMM (ALCYONE, CASSIOPEIA, MAIA) and three studies for RRMM (CANDOR, CASTOR, POLLUX). The generated hazard ratios (HRs) for each study closely aligned with reported HRs for high-risk patients, with a maximum relative error of less than 2%. By pooling these datasets, we obtained synthetic datasets for NDMM and RRMM patients. Stratified Cox regression analyses yielded HRs (95% confidence interval) of 0.764 (0.529-1.104) for NDMM and 0.431 (0.294-0.613) for RRMM. Random-effects meta-analyses showed HRs (95% confidence interval) of 0.764 (0.529-1.104) for NDMM and 0.425 (0.287-0.621) for RRMM. Compared with traditional meta-analysis techniques, the synthetic data approach provides additional clinical insights that were not available in the original publications. For instance, the clinical results for the three NDMM studies are summarized as KM curves in Figure 1, which were not reported in the original studies. Conclusions: Our results suggest that the addition of daratumumab to standard regimens improves PFS for RRMM patients with high-risk cytogenetic factors, although its benefit for NDMM patients is not statistically significant. Pooling synthetic data enhances the robustness of conventional meta-analysis outcomes. The identification of HRs and corresponding confidence intervals through SynIPD improves the credibility of our proposed method, offering a reliable approach to evaluating the efficacy of daratumumab in high-risk patients. Since SynIPD includes individual covariate information, it provides deep clinical insights beyond merely enhancing meta-analysis. In other words, any statistical results derived from the original IPD can be effectively approximated using SynIPD.

e13656 Background: Evidence-based medicine is crucial for clinical decision-making, yet studies suggest that a significant proportion of treatment decisions do not fully incorporate the latest evidence. Large Language Models (LLMs) show promise in bridging this gap, but their reliability for medical recommendations remains uncertain. Methods: We conducted an evaluation study comparing five LLMs’ recommendations across 50 clinical scenarios related to multiple myeloma diagnosis, staging, treatment, and management. The evaluation included three general-purpose LLMs (OpenAI o1-preview, Claude 3.5 Sonnet, Gemini 1.5 Pro), one retrieval-augmented generation (RAG) system (Myelo), and one agentic workflow-based system (HopeAI). General-purpose LLMs generated responses based solely on their internal knowledge, while the RAG system enhanced these capabilities by incorporating external knowledge retrieval. The agentic workflow system extended the RAG approach by implementing multi-step reasoning and coordinating with multiple tools and external systems for complex task execution. Three independent hematologist-oncologists evaluated the LLM-generated responses using standardized scoring criteria developed specifically for this study. Performance assessment encompassed five dimensions: accuracy, relevance, comprehensiveness, hallucination rate, and clinical use readiness. Results: HopeAI demonstrated superior performance across accuracy (82.0%), relevance (85.3%), and comprehensiveness (74.0%), compared to OpenAI o1-preview (64.7%, 57.3%, 36.0%), Claude 3.5 Sonnet (50.0%, 51.3%, 29.3%), Gemini 1.5 Pro (48.0%, 46.0%, 30.0%), and Myelo (58.7%, 56%, 32.7%). Hallucination rates were consistently low across all systems: HopeAI (5.3%), OpenAI o1-preview (3.3%), Claude 3.5 Sonnet (10.0%), Gemini 1.5 Pro (8.0%), and Myelo (5.3%). Clinical use readiness scores were relatively low for all systems: HopeAI (25.3%), OpenAI o1-preview (6.0%), Claude 3.5 Sonnet (2.7%), Gemini 1.5 Pro (4.0%), and Myelo (4.0%). Conclusions: This study demonstrates that while current LLMs show promise in medical decision support, their recommendations require careful clinical supervision to ensure patient safety and optimal care. Further research is needed to improve their clinical use readiness before integration into oncology workflows. These findings provide valuable insights into the capabilities and limitations of LLMs in oncology, guiding future research and development efforts toward integrating AI into clinical workflows.

7547 Background: Minimal residual disease (MRD) has been recently accepted by the Food and Drug Administration (FDA) as an endpoint for accelerated approval in Multiple Myeloma. However, emerging data from recent trials were not included in previous analyses. While literature-based meta-analyses on the correlation between MRD and approved clinical outcomes typically require extensive manual review, leveraging AI with expert-in-the-loop validation can efficiently generate reliable evidence from comprehensive clinical studies with up-to-date outcomes. Methods: An AI-assisted framework was developed to identify relevant studies and extract critical information via two independent objectives. The first objective examined trial-level associations, modeling treatment effects on MRD and clinical endpoints across patient populations using weighted least squares, with association strength measured by coefficients of determination (R²) and 95% confidence intervals (CIs). The second objective analyzed individual-level associations using synthetic individual patient data (SynthIPD) generated from the published Kaplan-Meier plots and summary statistics of patient subgroups. Results: AI-assisted screening identified eligible studies (>50 patients per treatment arm) reporting progression-free survival (PFS), overall survival (OS), and MRD-negative complete response rates (MRD-CR rate) using multi-parameter next-generation flow cytometry or sequencing methods (sensitivity threshold ≥10⁻⁵), expanding previous analyses from 15 to 20 two-arm studies. Trial-level analysis demonstrated an R² of 0.69 (95% CI 0.50–0.89) for PFS log hazard ratio versus MRD-CR rate log odds ratio. Analysis of synthetic individual data from Kaplan-Meier curves using a novel digitization method yielded a global odds ratio of 7.28 (95% CI 5.60–8.95) for individual-level correlation between MRD-CR rates and PFS outcomes. Conclusions: This study validates MRD-CR rate as an endpoint for accelerated approval in MM through rapid AI-assisted literature review and synthetic individual patient data. The findings demonstrate moderate correlation between MRD-CR rate and median PFS at both trial and individual levels, consistent with previous literature but incorporating additional eligible studies. The novel SynthIPD approach presents an efficient alternative to traditional data-sharing methods while maintaining analytical robustness. These results align with current Oncologic Drugs Advisory Committee (ODAC) surrogacy analysis methods and support the utility of MRD assessment in MM clinical trials.