Rubinstein, Sam

Introduction Registrational trials are clinical trials which are used to justify regulatory approval of a new medication by the FDA. However, clinical trials may not be designed around patient-centred outcomes, like overall survival (OS) or quality of life. This disconnect raises concerns about whether current trial designs are truly aligned with the long-term needs and priorities of patients. A comprehensive analysis of trial designs and endpoints is necessary to ensure that future oncological therapies deliver meaningful clinical improvements. Our objective was to describe types of clinical trial design and primary endpoints used for registrational trials, and their trends over time. Research design and methods In this cohort study, the HemOnc database was used to identify registrational trials supporting cancer drug approval from 2004 through 2023. Clinical trial design (escalation, de-escalation, in-class switch, out-of-class switch) and primary endpoint (OS; time-to-event including progression-free survival; response-based endpoint) were characterized. Results Escalation trial designs were more common than in-class switches or out-of-class switches. De-escalation and mixed designs were exceedingly rare. The most common primary endpoint was a composite time-to-event endpoint. Conclusion Most clinical trials leading to FDA approval for cancer-directed therapies employ escalation designs with composite time-to-event endpoints. Designs reflecting paradigm shifts in therapy (in-class switches and out-of-class switches) are sparsely used. This approach raises the concern that trials may increase therapeutic toxicity without delivering meaningful improvements in OS or quality of life.

e13656 Background: Evidence-based medicine is crucial for clinical decision-making, yet studies suggest that a significant proportion of treatment decisions do not fully incorporate the latest evidence. Large Language Models (LLMs) show promise in bridging this gap, but their reliability for medical recommendations remains uncertain. Methods: We conducted an evaluation study comparing five LLMs’ recommendations across 50 clinical scenarios related to multiple myeloma diagnosis, staging, treatment, and management. The evaluation included three general-purpose LLMs (OpenAI o1-preview, Claude 3.5 Sonnet, Gemini 1.5 Pro), one retrieval-augmented generation (RAG) system (Myelo), and one agentic workflow-based system (HopeAI). General-purpose LLMs generated responses based solely on their internal knowledge, while the RAG system enhanced these capabilities by incorporating external knowledge retrieval. The agentic workflow system extended the RAG approach by implementing multi-step reasoning and coordinating with multiple tools and external systems for complex task execution. Three independent hematologist-oncologists evaluated the LLM-generated responses using standardized scoring criteria developed specifically for this study. Performance assessment encompassed five dimensions: accuracy, relevance, comprehensiveness, hallucination rate, and clinical use readiness. Results: HopeAI demonstrated superior performance across accuracy (82.0%), relevance (85.3%), and comprehensiveness (74.0%), compared to OpenAI o1-preview (64.7%, 57.3%, 36.0%), Claude 3.5 Sonnet (50.0%, 51.3%, 29.3%), Gemini 1.5 Pro (48.0%, 46.0%, 30.0%), and Myelo (58.7%, 56%, 32.7%). Hallucination rates were consistently low across all systems: HopeAI (5.3%), OpenAI o1-preview (3.3%), Claude 3.5 Sonnet (10.0%), Gemini 1.5 Pro (8.0%), and Myelo (5.3%). Clinical use readiness scores were relatively low for all systems: HopeAI (25.3%), OpenAI o1-preview (6.0%), Claude 3.5 Sonnet (2.7%), Gemini 1.5 Pro (4.0%), and Myelo (4.0%). Conclusions: This study demonstrates that while current LLMs show promise in medical decision support, their recommendations require careful clinical supervision to ensure patient safety and optimal care. Further research is needed to improve their clinical use readiness before integration into oncology workflows. These findings provide valuable insights into the capabilities and limitations of LLMs in oncology, guiding future research and development efforts toward integrating AI into clinical workflows.

7533 Background: Maintenance treatment (rx) with lenalidomide (R) improves overall survival (OS) in multiple myeloma (MM). We report a retrospective cohort of patients (pts) with second primary B-cell acute lymphoblastic leukemia (ALL) diagnosed after R-rx for MM. Methods: We included pts who were exposed to at least 1 yr of R for MM rx prior to ALL dx. After approval by institutional review board, data were collected by chart review at participating centers using Redcap. OS was the primary endpoint. We used BlueSky statistics v10.3.1 for analysis. Results: Analysis included 33 ALL pts diagnosed from 2014-2022 at 7 sites. 52% pts were female. Median follow up was 9.6 yrs (95%CI,7.6-11.8) from MM dx and 3.2 yrs (CI, 2.1-4.7) from ALL dx. Pt characteristics at MM dx are given in Table. Induction rx included: proteasome inhibitor (PI) +immunomodulatory drug (IMiD) (43%), PI+alkylator (24%), IMiD (18%) & others (15%). 50% pts received autologous stem cell transplantation (ASCT). Median OS from MM dx was 11.2 yrs (CI, 8.9-NR). Time from MM dx and initial R rx to ALL dx were 5.0 yrs (range, 2.4-18.5) and 4.4 yrs (2.2-10.6) respectively. Median duration of R exposure before ALL dx was 3.8 yrs (1.6-10.4). Table shows characteristics of pts at B-ALL dx and genetic classification based on karyotyping (n=30), ALL FISH (n=28), chromosomal microarray (n=5) and next-generation sequencing (n=5). 31 pts received ALL-directed rx while 1 pt was initially managed with close observation as the pt attained CR on stopping R ( Lakshman et al; ASH 2019 ). Rx was unknown in 1 pt. Initial rx included: hyper-CVAD+/- rituximab- 49%, pediatric inspired protocols-18%, reduced intensity rx for frail pts- 21% and others- 12%. Response to first line rx were: CR- 56%, CRi- 28%, induction failure- 16%. 32% pts attained measurable residual disease (MRD) negativity. 48% pts underwent SCT for ALL. 14 pts died during follow up. 3-yr OS from ALL-dx was 50% (CI, 33%-76%). 3 pts required rx for MM after ALL diagnosis (2 had allo-SCT for ALL before MM relapse). Conclusions: Most pts had standard risk MM and were exposed to R for several years prior to dx of B-ALL, characterized by high incidence of hypodiploid karyotype; IKZF1 loss and TP53 deletions/mutations were detected in a subset of pts.MRD-adapted timely discontinuation of R in standard risk MM may decrease risk of second primary ALL. [Table: see text]

11079 Background: Phase 3 randomized clinical trials (RCTs) inform the standard of care in cancer treatment yet have a mixed track record of success. In this study, we sought to 1) determine the overall success rate of phase 3 RCTs; 2) assess whether authorship patterns differ between successful (positive) and unsuccessful (negative) trials; and 3) explore if individual authors have distinct track records of success. Methods: The HemOnc knowledgebase (KB) was queried for initial publications of phase 3 RCTs of systemic anticancer therapy; those without primary outcome described were excluded. RCTs were labeled positive if the primary endpoint was met per HemOnc standard criteria (P value ≤ 0.05 or hazard ratio upper bound ≤ 1 for any experimental arm[s]) and negative if they did not meet these criteria or if the experimental arm was statistically inferior. Trial metadata and author information were extracted from the HemOnc KB. Author positive publication rate (PPR) was defined as % of publications reporting a positive result divided by total number of eligible publications. Prolific authors were defined as those with 6+ eligible publications. Author count for manuscripts reporting positive vs negative results was evaluated with the Mann-Whitney U test; other statistics were descriptive. Results: 3328 studies met criteria as of February 5, 2024; these were associated with 3305 initial publications between 1960-2023, involving 25,412 unique authors. 1474 (44.3%) of the studies were positive. Publications reporting positive trial results had, on average, 3 more authors, median (interquartile range) of 18 (12-23) vs 15 (11-20), p<0.0001. The mean (±SD) PPR for n=1967 prolific authors was 47.7% (±22.3%). The PPR was not related to individual author output; mean PPR for non-prolific authors was 48.0% (SD not reported). Differences in PPR were apparent by the disease under study and the years of publication (Table). Among the 46 prolific authors who had a PPR 2+ SD above the mean, 26 (55%) were from China followed by Italy with 3 (6%). Conclusions: Despite known positive publication bias, the success rate of published phase 3 RCTs remains <50%. Publications reporting positive results had significantly more authors; this finding requires further study including adjustment for trial sample size. Significant variation in PPR amongst fields could be due to watershed therapeutic advances, differential selection of primary endpoints, and/or selective reporting of results. Finally, a signal for geographic association of outlier PPRs suggests the need for further investigation. [Table: see text]

11030 Background: Drug regulators operate on their own timelines to assess the safety and efficacy of new drugs and/or extensions to existing indications, such that there are delays between regulators. During such a gap, patients in the “early” domain have access to the approved drug, while those in the “late” domain do not, outside of compassionate use. Our study aimed to quantify the “opportunity lost” for new therapies approved by FDA and EMA, assuming patients in the late domain receive standard-of-care (SOC) therapy – which is presumably inferior to the new drug – during the gap. Methods: Anticancer drugs approved by both FDA and EMA were identified through the HemOnc knowledgebase. Inclusion criteria included: 1) drugs approved in both domains based on the same randomized clinical trial (RCTs) for the same indication; 2) a primary time-to-event endpoint; and 3) a quantitated median time-to-event for the primary endpoint. For each included trial, the approval gap between FDA and EMA in days and the ratio of approval gap to median control arm time-to-event duration was calculated. A ratio ≥2 implies that, on average, no late domain patients starting treatment in the first half of the gap would be expected to have access to the new drug prior to an event. Results: Of 60 eligible RCTs, 59 had calculable median event durations. 25 (42%) had a primary overall survival endpoint; the most common surrogate endpoint was progression-free survival (PFS; 29/59; 49%). The median (interquartile range [IQR]) approval gap was 186 (124-271) days, with FDA being the first approver in 56/59 (95%) cases. The median (IQR) ratio of approval gap to median control arm event duration was 0.63 (0.35-1.11). Drugs with ratio ≥2 (Table) were targeted or immunotherapies; 7/8 (88%) were approved with PFS endpoint; 5/8 (62.5%) were approved for second line or later. Conclusions: Our study shows that FDA approves most drugs prior to EMA and a nontrivial number of patients may experience progression or death events while awaiting access to anticancer therapies already approved in another domain. The analysis does not take into consideration scenarios in which the new therapy is inferior to SOC or delays in patient access (due to slow uptake; regulatory barriers [in Europe/UK]; or patient affordability [in US]). Further regulatory collaboration (e.g., Project Orbis) between FDA and EMA could help to reduce and potentially ameliorate this differential in access to new cancer drugs in the future. [Table: see text]

11023 Background: Randomized clinical trials (RCTs) in cancer typically compare experimental to control arm regimens. The choice of an appropriate control (ctrl) arm can have a major impact on the expected success of an RCT. The frequency with which cancer RCTs use control arms based on established dosing and scheduling protocols has not been described. Methods: The HemOnc knowledgebase (KB) was used to identify systemic anti-cancer therapy (SACT) regimen variants, defined as regimens with identical components that differ in dosing, scheduling, and/or route. Non-cancer, nonrandomized, and non-SACT studies were excluded. Study publication year was used to define standard variants as those evaluated in the experimental arm of a positive phase 3 RCT >1 year prior to publication as a control arm, regardless of cancer type or context of treatment. Study-variant dyads were evaluated to determine whether the variant was standard. Success rates of RCTs with standard vs non-standard control arms were evaluated with Fisher’s exact test. Results: 5221 studies were associated with ≥1 named variant in the HemOnc KB, as of 2024-02-06. After exclusions, there were 3511 study-variant dyads (2386 studies; 1714 variants). The 9 most common regimens are shown in the Table. The median (IQR) number of variants per regimen was 2 (1-3); carboplatin & paclitaxel (CP) had the most variants (n=33). Across all control arm study-variant dyads, 2228/3492 (64%) utilized non-standard variants. For example, 60/97 studies (62%) of docetaxel as control used the 75 mg/m2 q3wk variant after it was established in 2000, whereas n=27 others used 19 non-standard variants. Trials that used a standard control arm had a numerically higher success rate, 45% vs 42% (OR 1.10, 95% CI 0.92-1.32). Conclusions: Non-standard regimen variants are frequently used in cancer RCTs. Reasons for this could include toxicity, patient convenience, or emerging data from smaller studies that establish comparable efficacy or improved toxicity to standard variants. However, the a priori efficacy of non-standard control arms is less rigorously established, and trials with standard control arms might be more successful. Future directions include quantifying the granular differences between standard and non-standard variants and investigating whether certain types of non-standard variation (e.g. fewer cycles, lower doses) associate with trial success rates. [Table: see text]

LBA110 Background: There are limited data on COVID-19 in patients with cancer. We characterize the outcomes of patients with cancer and COVID-19 and identify potential prognostic factors. Methods: The COVID-19 and Cancer Consortium (CCC19) cohort study includes patients with active or prior hematologic or invasive solid malignancies reported across academic and community sites. Results: We included 1,018 cases accrued March-April 2020. Median age was 66 years (range, 18-90). Breast (20%) and prostate (16%) cancers were most prevalent; 43% of patients were on active anti-cancer treatment. At time of data analysis, 106 patients (10.4%) have died and 26% met the composite outcome of death, severe illness requiring hospitalization, and/or mechanical ventilation. In multivariable logistic regression analysis, independent factors associated with increased 30-day mortality were age, male sex, former smoking, ECOG performance status (2 versus 0/1: partially adjusted odds ratio (pAOR) 2.74, 95% CI 1.31-5.7; 3/4 versus 0/1: pAOR 5.34, 95% CI 2.44-11.69), active malignancy (stable/responding, pAOR 1.93, 95% CI 1.06-3.5; progressing, pAOR 3.79, 95% CI 1.78-8.08), and receipt of azithromycin and hydroxychloroquine. Tumor type, race/ethnicity, obesity, number of comorbidities, recent surgery, and type of active cancer therapy were not significant factors for mortality. Conclusions: All-cause 30-day mortality and severe illness in this cohort were significantly higher than previously reported for the general population and were associated with general risk factors as well as those unique to patients with cancer. Cancer type and treatment were not independently associated with increased 30-day mortality. Longer follow-up is needed to better understand the impact of COVID-19 on outcomes in patients with cancer, including the ability to continue specific cancer treatments.