Roose, James

Abstract When using real-world data to construct an external comparator arm for a single-arm trial, there may be differences in how and when patients are assessed for disease between trial and real-world settings. Such differences can generate outcome measurement error when comparing time-to-event endpoints and lead to biased findings. Recent methods have been developed to mitigate measurement error bias in real-world endpoints; however, they rely on the existence of a validation sample, ie, data on a set of patients where both the “true” trial-like and “mis-measured” real-world measures are collected. We demonstrate how novel statistical methods can be leveraged as quantitative bias analyses (QBA) to contextualize real-world evidence findings when outcome measurement error is of concern, but validation samples are infeasible to collect. Quantitative bias analyses allow researchers to set plausible ranges for the amount of error when not directly measurable. We highlight how to conduct QBA with two recent methods, cumulative incidence curve correction and survival regression calibration, and illustrate how to generate plausible parameter values through simulation. We provide an illustrative QBA example in a cohort of real-world patients with newly diagnosed multiple myeloma and provide practical guidance to apply QBA for outcome measurement error and interpret results.

Introduction: Progression-free survival (PFS), a common endpoint used in multiple myeloma (MM) trials, is defined as the time to the earliest occurrence of disease progression or death. Disease progression in MM is determined by the International Myeloma Working Group (IMWG) Uniform Response Criteria based on imaging and biomarkers from blood, urine, and bone marrow biopsy testing. In clinical trials, IMWG-recommended assessments are protocol-based. However, in routine care, they are often performed and recorded differently based on a multitude of factors. Algorithms to ascertain progression events using routinely collected biomarker data are important to classify disease progression and align with trial-defined endpoints. The aim of this study was to construct and evaluate an optimized real-world Progression (rwP) algorithm using real-world data (RWD) and demonstrate how biomarker data are captured and used to derive endpoints in routine practice. Method: A cohort of 270 transplant ineligible newly diagnosed MM (NDMM) patients starting on lenalidomide and dexamethasone (Rd) and meeting select MAIA (NCT02252172) trial eligibility criteria were included from the nationwide Flatiron Health electronic health record-derived deidentified database from 2015 to 2023. A rwP algorithm was developed using key IMWG-recommended biomarkers, including serum protein electrophoresis (SPEP), urine protein electrophoresis (UPEP), and serum free light chains (FLCs). The initial baseline value for each biomarker was identified, if available, during either the pre-treatment period (45 days prior through 7 days after first line treatment (1L) start) or the on-treatment period (8 days after 1L start through 1L end). Available biomarkers were followed for disease assessment. If any subsequent lab value during 1L treatment was less than the initial baseline value, the subsequent “on-treatment baseline” value was adopted as the new lowest value; otherwise, the initial value was used as the “on-treatment baseline” value for assessing progression events. Three hierarchies, which describe the order of utilizing key IMWG-recommended biomarkers to define rwP events, were explored, including: i) strict hierarchy (SH) [SPEP > UPEP > FLCs], ii) partial hierarchy (PH) [SPEP = UPEP > FLCs], and iii) no hierarchy (NH) [SPEP = UPEP = FLCs]. The PH option closely resembles the IMWG criteria, with FLCs only used for rwP assessments when no measurable SPEP/UPEP are identified. The NH approach considers all available biomarkers equally and resembles how they are utilized in real-world clinical practice. The feasibility of requiring confirmatory events following the first suggestion of progression was also explored. Assays following the progression event up to the end of 1L were considered as confirmatory progression events if they met IMWG criteria. Results: We applied the rwP algorithm with each hierarchy to all eligible NDMM patients (N = 270). The SH, PH, and NH approaches respectively identified 106 (39.3%), 106 (39.3%), and 123 (45.6%) patients with progression events. Results from the SH and PH approach are consistent likely due to the rarity of UPEP usage in clinical practice. The NH approach captured more progression events because FLCs are used in the presence of measurable SPEP or UPEP, resulting in 73 progression events captured by FLCs in comparison to 32 from other approaches, potentially increasing false-positive capture due to the volatility of FLC results. Among the 106 PH-identified patients who progressed, 84 (79.2%) had IMWG-recommended tests following their progression event, and 74 (69.8%) had their progression confirmed before the end of 1L, with 35 (33.0%) receiving their confirmatory reading within 2 months after progression. These results suggest that requiring confirmatory testing as part of the rwP is feasible, though it may introduce false negatives for patients receiving clinical progression confirmation instead of getting confirmatory labs in routine clinical practice. Conclusion: We present a comprehensive assessment of important design choices for an rwP algorithm for NDMM, including a hierarchy of key assays and the inclusion of confirmatory testing. Further validation of the algorithm with health care provider assessments and the inclusion of bone marrow biopsy and imaging data is important for evaluating the performance of the algorithm in RWD.

Introduction: Most induction regimens for multiple myeloma (MM) contain bortezomib. In most clinical trials, bortezomib has been prescribed either intravenously or subcutaneously twice per week in 21-day or 28-day cycles. However, several retrospective single-center studies have shown that once-weekly bortezomib has comparable efficacy and less peripheral neuropathy compared to twice-weekly bortezomib ( Sidana et al, PLoS One 2017; Cook et al, AJH 2021). These findings have also been confirmed by a secondary analysis of data from three prospective Phase 3 randomized studies ( Mateos et al, Haematologica 2014). As such, many clinicians have adopted once-weekly bortezomib regimens in routine care in the past decade. However, the real-world (RW) prevalence and efficacy of once-weekly vs twice-weekly bortezomib regimens in a larger population have not yet been characterized. Methods: This study used US nationwide longitudinal Flatiron Health electronic health record-derived de-identified data. We included patients aged 18-70 years with newly diagnosed MM between 2017-2022 with ≥ 6 months of follow-up and ≥ 2 clinical visits who were treated with a first-line (1L) bortezomib-containing regimen. Patients aged >70 were excluded a priori given the likelihood of altered practice patterns and more frequent comorbidities. Twice-weekly bortezomib was defined as patients for whom the most frequent interval between two doses of 1L bortezomib within a cycle was 3.0-4.9 days, while once-weekly bortezomib was defined as a corresponding interval of 5.0-9.9 days. We used multivariable logistic regression to assess demographic and clinical factors associated with once-weekly vs twice-weekly (reference) bortezomib. RW progression free survival (rwPFS) and RW overall survival (rwOS) from 1L treatment initiation were estimated using the Kaplan-Meier method. Outcomes were compared using Cox proportional hazards model including transplant as a time-varying covariate and adjustment for other demographic and clinical factors. Results: Of 2,522 patients included, 927 (36.8%) received twice-weekly while 1,522 (63.2%) received once-weekly bortezomib. Approximately half of patients (50.3%) were non-Latinx White, 21.7% non-Latinx Black, and 8.0% Latinx. Most patients were treated in the community setting (78.8%). Rates of once-weekly bortezomib administration were 66.2% and 62.1% among patients treated at academic and community practices, respectively. The frequency of weekly bortezomib use increased over time, from 58.0% between 2017-2019 to 68.8% between 2020-2022. In multivariate analysis, ECOG PS ≥ 2 (odds ratio [OR] 1.50; [95% CI 1.14 to 1.99] vs ECOG PS 0), and more recent year of diagnosis (OR 1.14; [95% CI 1.08 to 1.20] per year) were associated with higher odds of receiving once-weekly bortezomib. Conversely, Latinx patients vs non-Latinx White (OR 0.60 [95% CI 0.44 to 0.82]) and patients with ≥ 2 high-risk cytogenetic abnormalities (HRCA) vs 0 HRCA (OR 0.66 [95% CI 0.48 to 0.91]) had lower odds of receiving once-weekly bortezomib administration. No significant associations were found between gender, ISS stage, practice setting, insurance category, or baseline creatinine. There was no statistically significant difference in rwPFS among patients receiving once-weekly (median 37.3 months, 95% CI 33.8 to 42.5) vs twice-weekly bortezomib (median 39.2 months, 95% CI 32.9 to 42.8) (HR 0.87 [95% CI 0.77 to 1.00]) (Figure 1A). Similarly, there was no statistically significant difference in rwOS with median 25.5 months follow-up (Figure 1B). Median rwOS was not reached in either group (hazard ratio 0.89 [95% CI 0.75 to 1.07]). Discussion: We evaluated US bortezomib prescribing patterns for the first time in the modern era using a large nationwide RW-study. Whereas prescribing patterns have shifted towards once-weekly bortezomib over time, over 30% of patients diagnosed in the last three years received twice-weekly bortezomib. Importantly, once-weekly and twice-weekly bortezomib performed comparably with regard to rwPFS and rwOS. While our study did not analyze neuropathy or patient quality of life, once-weekly bortezomib is known to be associated with less neuropathy and less time in clinic. These data further support the conclusion that once-weekly bortezomib should be considered the standard of care for newly diagnosed patients with MM.

Abstract Background: Real-world data (RWD) derived from electronic health records are becoming increasingly important for deriving insights from clinical practice to complement findings from clinical trials. Response to treatment in multiple myeloma (MM) is assessed using the International Myeloma Working Group (IMWG) response criteria based on MM-specific laboratory measures (i.e. monoclonal [M] protein in serum and urine, free light chain [FLC] levels), as well as radiological images and bone marrow (BM) investigations when appropriate. As healthcare providers do not routinely collect all the information required by the IMWG response criteria, RWD data are often incomplete. Here, we present a derived real-world response (dR) algorithm based on IMWG criteria that accounts for lab measures routinely collected in the clinical care of MM patients and evaluate its ability to accurately assess response to treatment using clinical trial patient-level data. Methods: Treatment response is assessed using a rule-based algorithm integrating longitudinal laboratory measures routinely captured in RWD (Xu et al. Pharmacoepidemiol Drug Saf 2021) as e.g. by the Flatiron MM database. The rules are based on 'relaxed' IMWG criteria, which entail exclusion of BM biopsies data and of imaging results, and reduction in either serum or urine M protein levels (rather than both) to assign partial response. This algorithm was applied to patient-level clinical trial data from the Bellini trial (Kumar et al. Lancet Oncol 2020) with response assignment made by an independent review committee (IRC) in the trial used as the 'ground truth'. Agreement between the IRC's and algorithm's assignments of response was estimated using Cohen's Kappa statistic. Differences in overall response rate (ORR) between treatment and placebo arms were calculated using stratified Cochran-Mantel-Haenszel tests based on strata at randomization with number of prior lines of therapy and previous proteasome inhibitor treatment status variables used for stratification. Results: The Bellini trial is a Phase 3 clinical trial with 2:1 design enrolling 194 and 97 patients in its treatment and placebo arms, respectively (291 patients in total). Regular assessments of M protein and FLC levels were performed (median 13 or 14 measurements), providing detailed trajectory of patient response (Table 1). Comparison between the IRC's and algorithm's assignments of responders classified as Partial Response or better (PR+) resulted in almost perfect agreement with Cohen's Kappa 0.82 (274/291 assignments in agreement). It is worth noting that the Cohen's Kappa between the IRC's and Investigator's assessments is 0.85, indicating that the algorithm's error is within some expected uncertainty. Due to the exclusion of BM information, agreement decreased to 0.56 when depths of response were considered separately as opposed to grouping PR+ patients together with most cases being overestimated as complete response (CR) or stringent CR. In assessing treatment effect, differences in ORR between the intervention and placebo arms in the trial based on IRC's assessment could be accurately recapitulated (68% [66/97] vs 71% [69/97] responders in the placebo arm and 82% [159/194] vs 87% [169/194] in the intervention arm for IRC's and algorithm's assessment, respectively). Based on these results, algorithm response assignment led to consistent conclusions about treatment efficacy in the Bellini trial. Implementation of criteria to characterize very good partial response or better (VGPR+) also led to a conclusion consistent with the Bellini trial IRC's assessment, while implementation of criteria of CR+ resulted in estimates of treatment efficacy that were higher, but directionally consonant (Table 2). Conclusions: We present a fully automated rule-based algorithm for response assessment in MM relying on longitudinal lab measurements during treatment. The algorithm uses 'relaxed' IMWG criteria to account for routine clinical practice settings and demonstrates very high agreement with the assessment of trained clinicians, while reliably reproducing the efficacy analysis in the Bellini trial when ORR and VGPR+ are considered. We envision that, if the concordance observed here is confirmed in other independent cohorts, this algorithm can be used in the assessment of response in RWD and can facilitate response assessment when suboptimal amounts of data are available. Figure 1 Figure 1. Disclosures Tyanova: F. Hoffmann-La Roche: Current Employment. Xu: F. Hoffmann-La Roche AG: Current Employment. Williamson: Amgen: Current equity holder in publicly-traded company; Genentech: Current Employment, Current equity holder in publicly-traded company. Rocci: Novartis: Other: Wife is and employee of Novartis and holds Novartis stocks; Sanofi Aventis: Membership on an entity's Board of Directors or advisory committees; Janssen: Membership on an entity's Board of Directors or advisory committees, Other: Received travel sponsorship ; Janssen-Cilag Ltd: Honoraria, Speakers Bureau; Amgen: Honoraria, Membership on an entity's Board of Directors or advisory committees, Speakers Bureau; Celgene: Honoraria, Other: Received travel sponsorship ; Owner of 100% stocks of Harlock Healthcare Consulting Ltd (UK privately-held company currently not active).: Current holder of individual stocks in a privately-held company; Roche: Current equity holder in publicly-traded company; Takeda: Consultancy, Honoraria, Other: Received travel sponsorship , Speakers Bureau; Sanofi: Consultancy; NHS: Ended employment in the past 24 months; F. Hoffmann-La Roche Ltd.: Current Employment; AbbVie: Other: Received travel sponsorship . Hong: Imago BioSciences: Current Employment; Genentech, Inc.: Ended employment in the past 24 months; Stock options in Imago BioSciences: Current equity holder in publicly-traded company. Jin: Genentech Inc: Current Employment; Roche: Current equity holder in publicly-traded company. Karve: AbbVie: Current Employment, Current equity holder in publicly-traded company. Roose: Flatiron Health, Inc.: Current Employment; Roche: Current equity holder in publicly-traded company. Sawas: Seattle Genetics: Honoraria; Affimed: Research Funding; Roche: Current equity holder in publicly-traded company; Flat Iron Health: Current Employment; Acrotech: Honoraria; Daiichi-Sankyo: Speakers Bureau; Seattle Genetics: Speakers Bureau; Gilead: Speakers Bureau. Kumar: Antengene: Consultancy, Honoraria; Novartis: Research Funding; Adaptive: Membership on an entity's Board of Directors or advisory committees, Research Funding; Astra-Zeneca: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Carsgen: Research Funding; Tenebio: Research Funding; Beigene: Consultancy; Oncopeptides: Consultancy; KITE: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Celgene: Membership on an entity's Board of Directors or advisory committees, Research Funding; Merck: Research Funding; Bluebird Bio: Consultancy; Roche-Genentech: Consultancy, Research Funding; Janssen: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Takeda: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Amgen: Consultancy, Research Funding; BMS: Consultancy, Research Funding; Abbvie: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Sanofi: Research Funding.