Dahm, Philipp

Abstract Introduction While certainty of evidence assessment is key to a rigorous and transparent systematic review, it is unknown how – and how frequently – it is assessed in systematic reviews. The objective of this study was to examine the prevalence and approaches used for certainty of evidence assessment in systematic reviews published in high‐impact medicine journals over the past 11 years. Methods A PubMed search and hand‐searching of relevant journal websites identified systematic reviews published between 24 January 2013 and 23 January 2024 in any of the ten highest‐impact journals in the General and Internal Medicine category of the Journal Citation Report. Two reviewers independently selected any systematic review related to health outcomes assessing certainty of evidence using any method. We extracted data related to review characteristics, certainty of evidence and risk of bias/methodological quality assessment frameworks, and reported consideration of certainty of evidence domains. Logistic regression examined year of publication to determine whether the prevalence of certainty of evidence assessment changed over time. Results Of 1,023 included reviews, 346 (33.8%) assessed certainty of evidence. Prevalence of certainty of evidence assessment increased over time (0.16 ± 0.2; p < .001). Most (89.3%) of reviews used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess certainty of evidence. Conclusion Only one in three systematic reviews published in the highest‐impact medical journals over the past 11 years assessed certainty of evidence, though prevalence increased over time. The use of specific domains within each certainty of evidence framework was not clearly described in all reviews.

When one initially targets the null effect and the point estimate falls close to the null, two challenges exist in rating certainty of evidence. First, when the point estimate is near the null and the data, therefore, suggests little or no effect, rating certainty in a benefit or harm is misleading. Second, since in general the narrower the confidence interval (CI) the more precise the estimate, if the CI is narrow, rating down for imprecision due simply to crossing the null is inappropriate. This paper addresses these issues and provides a solution: to revise the target of certainty rating from a non-zero effect to a little or no effect. This solution requires estimating a range in which the minimal important difference (MID) for benefit and an MID for harm might lie, and thus establishing a range that represents little or no effect. If GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) users are confident that the point estimate represents an effect less than the smallest plausible MID, they will revise their target and rate certainty to a little or no effect. If the entire CI falls within the range of little or no effect, they will not rate down for imprecision. Otherwise (if the CI includes an important effect), they will rate down. Using the solution provided in this paper GRADE users can make an optimal choice of the target of certainty rating.

Objectives To assess the effects of percutaneous nephrolithotomy (PCNL) vs retrograde intrarenal surgery (RIRS) for the treatment of renal stones in adults. Methods We performed a comprehensive search of the Cochrane Library, MEDLINE, Embase, three other databases, trials registries, other sources of the grey literature, and conference proceedings up to 23 March 2023. We applied no restrictions on publication language or status. Screening, data extraction, risk‐of‐bias assessment, and certainty of evidence (CoE) rating using the Grading of Recommendations Assessment, Development and Evaluations (GRADE) approach were done in duplicate by two independent reviewers. This co‐publication focuses on the primary outcomes of this review only. Results We included 42 trials that met the inclusion criteria. Stone‐free rate (SFR): PCNL may improve SFRs (risk ratio [RR] 1.13, 95% confidence interval [CI] 1.08–1.18; I 2 = 71%; 39 studies, 4088 participants; low CoE). Major complications: PCNL probably has little to no effect on major complications (RR 0.86, 95% CI 0.59–1.25; I 2 = 15%; 34 studies, 3649; participants; moderate CoE) compared to RIRS. Need for secondary interventions: PCNL may reduce the need for secondary interventions (RR 0.31, 95% CI 0.17–0.55; I 2 = 61%; 21 studies, 2005 participants; low CoE) compared to RIRS. Conclusion Despite shortcomings in most studies that lowered our certainty in the estimates of effect to mostly very low or low, we found that PCNL may improve SFRs and reduce the need for secondary interventions while not impacting major complications. Ureteric stricture rates may be similar compared to RIRS. We expect the findings of this review to be helpful for shared decision‐making about management choices for individuals with renal stones.

While numerous guidelines for the prescription of physical activity are released each year, the quality and practicability of these guidelines is unknown. We assessed the quality of 95 guidance documents published since 2000 that included recommendations about physical activity for the promotion of general health and prevention of cardiometabolic disease. We used 3 tools: Appraisal of Guidelines for Research and Evaluation (AGREE II), the National Academy of Medicine’s (NAM) Standards for Trustworthy Clinical Practice Guidelines, and the Frequency, Intensity, Time, and Type (FITT) score. Average AGREE II domain scores ranged from 38%−84%, and the portion of criteria fulfilled per NAM domain ranged from 7%–39%. The average FITT score for all recommendations was 2.48 out of 4. While guidelines improved according to both AGREE II and the NAM standards over time, their practicability as assessed by FITT score did not improve. Guidelines produced by governmental agencies or other nonprofit organizations, using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach, or fulfilling a higher number of NAM criteria tended to be of higher quality. Organizations producing physical activity guidelines can improve their quality by establishing and reporting processes for public representation, external review, and conflict of interest (COI) management. Future recommendations about physical activity should be more specific and include strategies to improve implementation. Registration no.: PROSPERO CRD42019126364. Novelty: Most physical activity recommendations are not sufficiently specific to be practically implemented. The overall quality of guidelines has improved over time, but the specificity of recommendations has not. Improved public representation, external review, and COI disclosure and management processes would improve guideline quality.

Introduction Physical activity can prevent a wide range of diseases, including highly prevalent conditions such as heart disease, diabetes and associated cardiometabolic disorders. Numerous guidelines for the prescription of physical activity and exercise to promote general health and prevent disease are released each year, but the quality of these guidelines is currently unknown. This systematic review and critical appraisal of physical activity and exercise guidelines aims to summarise the current status and quality of these guidelines to provide suggestions to improve the development of future guidelines in this area. Methods and analysis We will conduct a systematic review of guidelines in Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations, and Daily, Ovid EMBASE, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews and Scopus databases published from database 2000 through 23 October 2019, written in English for the use of physical activity and exercise for the prevention of cardiometabolic disease and related risk factors in otherwise healthy individuals. We will also search the grey literature for additional eligible documents. We will use the Appraisal of Guidelines for Research and Evaluation II tool to assess the quality of eligible recommendations from all included guidelines, as well as perform exploratory analyses on guideline development variables. Ethics and dissemination As a protocol for the review and critical appraisal of published documents, no potential ethical considerations are discussed. The protocol will guide the development of the review, which will be disseminated to relevant journals for publication. PROSPERO registration number CRD42019126364