Lin, Timothy A.

PURPOSE Most oncology trials define superiority according to dichotomized P value thresholds, which are frequently misinterpreted. Posterior probability, however, directly estimates the probability of the hypothesis at hand. Here, we reanalyze a large collection of modern phase III trials and benchmark posterior probability versus the standard trial interpretation based on statistical significance. METHODS Outcomes from 194,129 patients were manually reconstructed from the primary end points of 230 phase III, superiority-design oncology trials. Posterior probabilities of treatment effect were then calculated across multiple priors and several effect sizes of clinical relevance, including minimum clinically important difference (MCID) defined as hazard ratio (HR) < 0.8 per ASCO criteria or HR < 0.64 per European Society of Medical Oncology (ESMO) criteria. RESULTS All trials interpreted as superior using P value thresholds had probabilities >90% for achieving at least marginal benefits (HR < 1). However, only 62% of positive trials (74/120) had >90% probabilities of achieving the ASCO MCID (HR < 0.8), even under an enthusiastic prior, including 70% of trials (57/82) leading to regulatory approval. Only 30% of positive trials (36/120) had >90% probability of achieving the ESMO MCID (HR < 0.64). Conversely, 24% of trials (26/110) interpreted as not superior had >90% probability of achieving marginal benefits (HR < 1), even under a skeptical prior. CONCLUSION Bayesian models, although often in agreement with statistical significance thresholds, add considerable unique interpretative value for a subset of phase III oncology trials. Posterior probability may provide a solution for overcoming the discrepancies between refuting the null hypothesis and detecting clinically relevant effects.

Abstract Background In randomized clinical trials (RCTs), anticipating control arm outcomes is important for power calculations and enrollment goals. Unexpected overperformance of the control arm can underpower RCTs. The objective of this study was to estimate the prevalence of control arm overperformance in phase 3 oncology RCTs. Methods The authors conducted a meta‐epidemiological study of two‐arm, superiority‐design, phase 3 oncology RCTs. They reviewed articles/publications and protocols for power calculations, justification, and outcomes. Control arm performance was measured as the ratio of observed control arm outcomes relative to pretrial estimates, and overperformance was defined as a 10% improvement. Results In total, 385 RCTs met inclusion criteria. The primary end point was overall survival (OS) in 134 RCTs (35%). Of 311 RCTs (81%) that provided a pretrial estimate for control arm performance, 138 RCTs (44%) cited a justification for the estimate. Greater than expected control arm performance was associated with lower odds of meeting the primary end point (adjusted odds ratio, 0.989; 95% confidence interval, 0.981–0.998; p = 0.015). Of 244 RCTs that were evaluable for control arm performance, 104 (43%) exhibited overperformance. RCTs that used OS as a primary end point were more likely to exhibit control arm overperformance compared with RCTs that used other primary end points (56% vs. 33%; adjusted odds ratio, 2.43; 95% confidence interval, 1.42–4.15; p = .001). Conclusions Control arm overperformance is common in phase 3 oncology RCTs and is associated with trial outcomes, suggesting an effect on study power. Because overperformance appears to be especially prevalent when OS is the primary end point, the effects of postprogression therapies on OS may be underappreciated. Alternative strategies for designing trials should be considered for reliable estimation of treatment effects.

Abstract In phase 3 oncology clinical trials, identifying and understanding potential sources of differential censoring (DC), which describes censoring imbalance between treatment arms, in overall survival (OS) endpoints is vital to reduce bias and misinterpretation of OS benefit. Among 322 trials, 34 (11%) trials exhibited DC overrepresented in the control arm (DCC). Compared to trials without DC or those with DC overrepresented in the experimental arm (DCE), trials with DCC showed higher rates of OS benefit (56% vs. 32%; odds ratio [OR] = 2.26, 95% CI = 1.04 to 4.90; P = .04) and were associated with patient withdrawal (median 8.0% vs. 3.8%; P = .0002), lack of double blinding (15% vs. 48%, OR = 0.19, 95% CI = 0.06 to 0.56; P = .003), and better global quality of life in the experimental arm (32% vs. 13%, OR = 3.01, 95% CI = 1.26 to 7.19; P = .01). OS DCC is a prominent challenge that requires rigorous trial design and comprehensive patient and mortality follow-up to reduce bias in interpreting OS benefit.

ABSTRACT Background Blinding mitigates bias in randomized trials and may be especially crucial for surrogate endpoints, such as progression‐free survival (PFS). Here, we characterize utilization of and factors associated with blinding in Phase III oncology trials with PFS primary endpoints. Methods Two‐arm, superiority‐design trials investigating systemic therapy were identified in May 2024 from ClinicalTrials.gov with no date limitation. Trials were required to have a PFS primary endpoint. The study outcomes were the presence of double‐blind designs and blinded independent central review (BICR) for the primary endpoint. Ninety‐five percent credible intervals for binary outcomes were estimated from beta distributions, and multivariable logistic regressions explored associations with trial‐level features. Results After screening, 237 trials were included, enrolling 127,518 patients. A double‐blind design was used in 105 trials (44%, 95% CrI 38%–51%). BICR was used in 111 trials (47%, 95% CrI 41%–53%), including 39 double‐blind trials (16%, 95% CrI 12%–22%). Trials with BICR had higher odds of meeting the primary endpoint (OR 1.84; 95% CI 1.06–3.18; p = 0.03). The PFS assessor identity (central vs. local) or blinding status was not reported in 50 trials (26%, 95% CrI 16%–27%). Trials that met prespecified significance criteria for PFS were more likely to report whether PFS assessors were blinded/unblinded and central/local (OR, 3.05; 95% Cl: 1.60–5.81; p = 0.0007). Conclusions Despite the importance of double blinding in combination with BICR for reducing bias, only a few trials blinded physicians, patients, and primary endpoint assessors. This meta‐epidemiological study illuminates the prevalence of potential assessment biases affecting PFS in Phase III oncology and secondarily emphasizes the need for improved methodology reporting.