Schneeweiß, Sebastian

Abstract When using real-world data to construct an external comparator arm for a single-arm trial, there may be differences in how and when patients are assessed for disease between trial and real-world settings. Such differences can generate outcome measurement error when comparing time-to-event endpoints and lead to biased findings. Recent methods have been developed to mitigate measurement error bias in real-world endpoints; however, they rely on the existence of a validation sample, ie, data on a set of patients where both the “true” trial-like and “mis-measured” real-world measures are collected. We demonstrate how novel statistical methods can be leveraged as quantitative bias analyses (QBA) to contextualize real-world evidence findings when outcome measurement error is of concern, but validation samples are infeasible to collect. Quantitative bias analyses allow researchers to set plausible ranges for the amount of error when not directly measurable. We highlight how to conduct QBA with two recent methods, cumulative incidence curve correction and survival regression calibration, and illustrate how to generate plausible parameter values through simulation. We provide an illustrative QBA example in a cohort of real-world patients with newly diagnosed multiple myeloma and provide practical guidance to apply QBA for outcome measurement error and interpret results.

Introduction: Progression-free survival (PFS), a common endpoint used in multiple myeloma (MM) trials, is defined as the time to the earliest occurrence of disease progression or death. Disease progression in MM is determined by the International Myeloma Working Group (IMWG) Uniform Response Criteria based on imaging and biomarkers from blood, urine, and bone marrow biopsy testing. In clinical trials, IMWG-recommended assessments are protocol-based. However, in routine care, they are often performed and recorded differently based on a multitude of factors. Algorithms to ascertain progression events using routinely collected biomarker data are important to classify disease progression and align with trial-defined endpoints. The aim of this study was to construct and evaluate an optimized real-world Progression (rwP) algorithm using real-world data (RWD) and demonstrate how biomarker data are captured and used to derive endpoints in routine practice. Method: A cohort of 270 transplant ineligible newly diagnosed MM (NDMM) patients starting on lenalidomide and dexamethasone (Rd) and meeting select MAIA (NCT02252172) trial eligibility criteria were included from the nationwide Flatiron Health electronic health record-derived deidentified database from 2015 to 2023. A rwP algorithm was developed using key IMWG-recommended biomarkers, including serum protein electrophoresis (SPEP), urine protein electrophoresis (UPEP), and serum free light chains (FLCs). The initial baseline value for each biomarker was identified, if available, during either the pre-treatment period (45 days prior through 7 days after first line treatment (1L) start) or the on-treatment period (8 days after 1L start through 1L end). Available biomarkers were followed for disease assessment. If any subsequent lab value during 1L treatment was less than the initial baseline value, the subsequent “on-treatment baseline” value was adopted as the new lowest value; otherwise, the initial value was used as the “on-treatment baseline” value for assessing progression events. Three hierarchies, which describe the order of utilizing key IMWG-recommended biomarkers to define rwP events, were explored, including: i) strict hierarchy (SH) [SPEP > UPEP > FLCs], ii) partial hierarchy (PH) [SPEP = UPEP > FLCs], and iii) no hierarchy (NH) [SPEP = UPEP = FLCs]. The PH option closely resembles the IMWG criteria, with FLCs only used for rwP assessments when no measurable SPEP/UPEP are identified. The NH approach considers all available biomarkers equally and resembles how they are utilized in real-world clinical practice. The feasibility of requiring confirmatory events following the first suggestion of progression was also explored. Assays following the progression event up to the end of 1L were considered as confirmatory progression events if they met IMWG criteria. Results: We applied the rwP algorithm with each hierarchy to all eligible NDMM patients (N = 270). The SH, PH, and NH approaches respectively identified 106 (39.3%), 106 (39.3%), and 123 (45.6%) patients with progression events. Results from the SH and PH approach are consistent likely due to the rarity of UPEP usage in clinical practice. The NH approach captured more progression events because FLCs are used in the presence of measurable SPEP or UPEP, resulting in 73 progression events captured by FLCs in comparison to 32 from other approaches, potentially increasing false-positive capture due to the volatility of FLC results. Among the 106 PH-identified patients who progressed, 84 (79.2%) had IMWG-recommended tests following their progression event, and 74 (69.8%) had their progression confirmed before the end of 1L, with 35 (33.0%) receiving their confirmatory reading within 2 months after progression. These results suggest that requiring confirmatory testing as part of the rwP is feasible, though it may introduce false negatives for patients receiving clinical progression confirmation instead of getting confirmatory labs in routine clinical practice. Conclusion: We present a comprehensive assessment of important design choices for an rwP algorithm for NDMM, including a hierarchy of key assays and the inclusion of confirmatory testing. Further validation of the algorithm with health care provider assessments and the inclusion of bone marrow biopsy and imaging data is important for evaluating the performance of the algorithm in RWD.