Chu, Haitao

Introduction Increasing publication of fraudulent clinical trials poses a serious threat to evidence-based medicine. In the VITALITY Study I, we demonstrated that contamination by retracted trials significantly distorts evidence synthesis. The upcoming VITALITY Study II will take a step further and investigate how such problematic evidence undermines the credibility of guideline recommendations. Methods and analyses The VITALITY Study II will adopt a retrospective cohort design and will be structured as three work packages (WPs). To start with, a cohort of clinical practice guidelines (CPGs) that were contaminated by retracted trials and/or meta-analyses that included retracted trials will be established based on forward citation searching (WP1). Then, recommendations from these CPGs that used evidence from retracted trials and/or meta-analyses that synthesised these retracted trials will be re-evaluated after removing such problematic evidence, in terms of both the direction and strength of effect sizes (WP2). Finally, the subsequent impact on patient outcomes attributable to distorted recommendations will be estimated on a hypothetical population, measured by the number of expected benefit loss and the number of expected harm increment per 100 000 patients (WP3). Ethics and dissemination Formal ethical approval is not necessary for this study as it does not involve human or animal participants, nor does it involve the collection of primary data. We will disseminate the findings through publication in peer-reviewed journals and, whenever possible, presentations at academic conferences.

AbstractMeta‐analysis is commonly used to combine results from multiple clinical trials, but traditional meta‐analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently‐introduced causally interpretable meta‐analysis methods and apply their treatment effect estimators to two individual‐participant data sets. These estimators transport estimated treatment effects from studies in the meta‐analysis to a specified target population using the individuals' potentially effect‐modifying covariates. We consider different regression and weighting methods within this approach and compare the results to traditional aggregated‐data meta‐analysis methods. In our applications, certain versions of the causally interpretable methods performed somewhat better than the traditional methods, but the latter generally did well. The causally interpretable methods offer the most promise when covariates modify treatment effects and our results suggest that traditional methods work well when there is little effect heterogeneity. The causally interpretable approach gives meta‐analysis an appealing theoretical framework by relating an estimator directly to a specific population and lays a solid foundation for future developments.

Abstract In a large cohort of United States healthcare personnel without prior coronavirus disease 2019 (COVID-19) infection, 94 382 doses of messenger RNA (mRNA) COVID-19 vaccine were administered to 49 220 individuals. The adjusted vaccine effectiveness following 2 doses of each of the 2 available brands of mRNA vaccine exceeded 96%.

Often clinicians are interested in determining whether a subject's measurement falls within a normal range, defined as a range of values of a continuous outcome which contains some proportion (eg, 95%) of measurements from a healthy population. Several studies in the biomedical field have estimated reference ranges based on a meta‐analysis of multiple studies with healthy individuals. However, the literature currently gives no guidance about how to estimate the reference range of a new subject in such settings. Instead, meta‐analyses of such normative range studies typically report the pooled mean as a reference value, which does not incorporate natural variation across healthy individuals in different studies. We present three approaches to calculating the normal reference range of a subject from a meta‐analysis of normally or lognormally distributed outcomes: a frequentist random effects model, a Bayesian random effects model, and an empirical approach. We present the results of a simulation study demonstrating that the methods perform well under a variety of scenarios, though users should be cautious when the number of studies is small and between‐study heterogeneity is large. Finally, we apply these methods to two examples: pediatric time spent awake after sleep onset and frontal subjective postural vertical measurements.

Publication bias occurs when studies with statistically significant results have increased likelihood of being published. Publication bias is commonly associated with inflated treatment effect which lowers the certainty of decision makers about the evidence. In this guide we propose that systematic reviewers and decision makers consider the direction and magnitude of publication bias, as opposed to just the binary determination of the presence of this bias, before lowering their certainty in the evidence. Direction of bias may not always exaggerate the treatment effect. The presence of bias with a trivial magnitude may not affect the decision at hand. Various statistical approaches are available to determine the direction and magnitude of publication bias.