Kouzy, Ramez

PURPOSE Most oncology trials define superiority according to dichotomized P value thresholds, which are frequently misinterpreted. Posterior probability, however, directly estimates the probability of the hypothesis at hand. Here, we reanalyze a large collection of modern phase III trials and benchmark posterior probability versus the standard trial interpretation based on statistical significance. METHODS Outcomes from 194,129 patients were manually reconstructed from the primary end points of 230 phase III, superiority-design oncology trials. Posterior probabilities of treatment effect were then calculated across multiple priors and several effect sizes of clinical relevance, including minimum clinically important difference (MCID) defined as hazard ratio (HR) < 0.8 per ASCO criteria or HR < 0.64 per European Society of Medical Oncology (ESMO) criteria. RESULTS All trials interpreted as superior using P value thresholds had probabilities >90% for achieving at least marginal benefits (HR < 1). However, only 62% of positive trials (74/120) had >90% probabilities of achieving the ASCO MCID (HR < 0.8), even under an enthusiastic prior, including 70% of trials (57/82) leading to regulatory approval. Only 30% of positive trials (36/120) had >90% probability of achieving the ESMO MCID (HR < 0.64). Conversely, 24% of trials (26/110) interpreted as not superior had >90% probability of achieving marginal benefits (HR < 1), even under a skeptical prior. CONCLUSION Bayesian models, although often in agreement with statistical significance thresholds, add considerable unique interpretative value for a subset of phase III oncology trials. Posterior probability may provide a solution for overcoming the discrepancies between refuting the null hypothesis and detecting clinically relevant effects.

Abstract Background In randomized clinical trials (RCTs), anticipating control arm outcomes is important for power calculations and enrollment goals. Unexpected overperformance of the control arm can underpower RCTs. The objective of this study was to estimate the prevalence of control arm overperformance in phase 3 oncology RCTs. Methods The authors conducted a meta‐epidemiological study of two‐arm, superiority‐design, phase 3 oncology RCTs. They reviewed articles/publications and protocols for power calculations, justification, and outcomes. Control arm performance was measured as the ratio of observed control arm outcomes relative to pretrial estimates, and overperformance was defined as a 10% improvement. Results In total, 385 RCTs met inclusion criteria. The primary end point was overall survival (OS) in 134 RCTs (35%). Of 311 RCTs (81%) that provided a pretrial estimate for control arm performance, 138 RCTs (44%) cited a justification for the estimate. Greater than expected control arm performance was associated with lower odds of meeting the primary end point (adjusted odds ratio, 0.989; 95% confidence interval, 0.981–0.998; p = 0.015). Of 244 RCTs that were evaluable for control arm performance, 104 (43%) exhibited overperformance. RCTs that used OS as a primary end point were more likely to exhibit control arm overperformance compared with RCTs that used other primary end points (56% vs. 33%; adjusted odds ratio, 2.43; 95% confidence interval, 1.42–4.15; p = .001). Conclusions Control arm overperformance is common in phase 3 oncology RCTs and is associated with trial outcomes, suggesting an effect on study power. Because overperformance appears to be especially prevalent when OS is the primary end point, the effects of postprogression therapies on OS may be underappreciated. Alternative strategies for designing trials should be considered for reliable estimation of treatment effects.

Abstract In phase 3 oncology clinical trials, identifying and understanding potential sources of differential censoring (DC), which describes censoring imbalance between treatment arms, in overall survival (OS) endpoints is vital to reduce bias and misinterpretation of OS benefit. Among 322 trials, 34 (11%) trials exhibited DC overrepresented in the control arm (DCC). Compared to trials without DC or those with DC overrepresented in the experimental arm (DCE), trials with DCC showed higher rates of OS benefit (56% vs. 32%; odds ratio [OR] = 2.26, 95% CI = 1.04 to 4.90; P = .04) and were associated with patient withdrawal (median 8.0% vs. 3.8%; P = .0002), lack of double blinding (15% vs. 48%, OR = 0.19, 95% CI = 0.06 to 0.56; P = .003), and better global quality of life in the experimental arm (32% vs. 13%, OR = 3.01, 95% CI = 1.26 to 7.19; P = .01). OS DCC is a prominent challenge that requires rigorous trial design and comprehensive patient and mortality follow-up to reduce bias in interpreting OS benefit.