Citations

Abstract Introduction While certainty of evidence assessment is key to a rigorous and transparent systematic review, it is unknown how – and how frequently – it is assessed in systematic reviews. The objective of this study was to examine the prevalence and approaches used for certainty of evidence assessment in systematic reviews published in high‐impact medicine journals over the past 11 years. Methods A PubMed search and hand‐searching of relevant journal websites identified systematic reviews published between 24 January 2013 and 23 January 2024 in any of the ten highest‐impact journals in the General and Internal Medicine category of the Journal Citation Report. Two reviewers independently selected any systematic review related to health outcomes assessing certainty of evidence using any method. We extracted data related to review characteristics, certainty of evidence and risk of bias/methodological quality assessment frameworks, and reported consideration of certainty of evidence domains. Logistic regression examined year of publication to determine whether the prevalence of certainty of evidence assessment changed over time. Results Of 1,023 included reviews, 346 (33.8%) assessed certainty of evidence. Prevalence of certainty of evidence assessment increased over time (0.16 ± 0.2; p < .001). Most (89.3%) of reviews used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess certainty of evidence. Conclusion Only one in three systematic reviews published in the highest‐impact medical journals over the past 11 years assessed certainty of evidence, though prevalence increased over time. The use of specific domains within each certainty of evidence framework was not clearly described in all reviews.

When one initially targets the null effect and the point estimate falls close to the null, two challenges exist in rating certainty of evidence. First, when the point estimate is near the null and the data, therefore, suggests little or no effect, rating certainty in a benefit or harm is misleading. Second, since in general the narrower the confidence interval (CI) the more precise the estimate, if the CI is narrow, rating down for imprecision due simply to crossing the null is inappropriate. This paper addresses these issues and provides a solution: to revise the target of certainty rating from a non-zero effect to a little or no effect. This solution requires estimating a range in which the minimal important difference (MID) for benefit and an MID for harm might lie, and thus establishing a range that represents little or no effect. If GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) users are confident that the point estimate represents an effect less than the smallest plausible MID, they will revise their target and rate certainty to a little or no effect. If the entire CI falls within the range of little or no effect, they will not rate down for imprecision. Otherwise (if the CI includes an important effect), they will rate down. Using the solution provided in this paper GRADE users can make an optimal choice of the target of certainty rating.

The optimal patient panel size (PPS) in primary care and the factors determining it remain unclear. We conducted a meta-narrative review of the literature to evaluate factors influencing PPS and assess its association with patient outcomes. A comprehensive search of electronic databases was performed from inception through December 2023, focusing on original studies reporting factors used to determine PPS and related outcomes (eg, clinical outcomes, process measures, and resource utilization). A total of 48 studies were included, identifying 7 key factors influencing PPS. Smaller panels were associated with improved patient satisfaction, continuity of care, and health promotion, while clinical outcomes, utilization, and costs showed minimal impact by PPS. Panel size was primarily associated with patient age, sex, comorbidities, and practice type and structure. Community-based centers typically managed smaller panels, often staffed by female clinicians and serving socioeconomically disadvantaged populations with greater health needs than hospital-based practices. Female clinicians were also independently associated with managing smaller panels, higher quality care indicators, fewer emergency department visits, and improved patient satisfaction. Determining the ideal PPS is a multifaceted process influenced by practice setting, patient demographics, and clinician characteristics. While practice-related factors showed limited association with PPS, patient-reported outcomes were more closely linked to it. Primary care practices should tailor panel sizes to their patient populations, emphasizing a patient-centered approach and ensuring adequate infrastructure support to optimize care delivery.

Abstract Globally, an estimated 300 million individuals have sickle cell trait (SCT), the carrier state for sickle cell disease (SCD). Although SCD is associated with increased morbidity and shortened life span, SCT has a life span comparable with that of the general population. However, “sickle cell crisis” has been used as a cause of death for decedents with SCT in reports of exertion-related death in athletes, military personnel, and individuals in police custody. To appraise this practice, the American Society of Hematology convened an expert panel of hematologists and forensic pathologists to conduct a systematic review of the literature relating to the occurrence of sickle cell pain crises and exertion-related mortality in people with SCT. Multiple bibliographic databases were searched with controlled vocabulary and keywords related to “sickle cell trait,” “vaso-occlusive pain,” and “death,” yielding 18 of 1474 citations. Independent pairs of reviewers selected studies and extracted data. We found no studies comparing uncomplicated acute pain crises in individuals with SCT and SCD. Additionally, no study was identified to support the occurrence of acute vaso-occlusive pain crises in individuals with SCT. Furthermore, this systematic review did not identify any evidence to support an association between SCT and sudden unexplained death in the absence of exertion-related rhabdomyolysis. We conclude that there are no data to support the diagnosis of acute vaso-occlusive sickle cell crisis as a cause of death in SCT, nor does the available evidence support the use of SCT as a cause of exertion-related death without rhabdomyolysis.

Abstract The common practice in meta‐analysis and clinical practice guidelines is to derive the absolute treatment effect (also called risk difference, RD) from a combination of a pooled relative risk (RR) that resulted from a meta‐analysis, and a user‐provided baseline risk (BR). However, this method does not address the uncertainty in BR. We developed a web‐based R Shiny tool to perform simple microsimulation and incorporate uncertainty in BR into the precision of RD. We empirically evaluated this approach by estimating the impact of incorporating this uncertainty when BR is derived from the control group rates in 3,128 meta‐analyses curated from the Cochrane Library (26,964 individual studies). When BR was derived from the largest study in each meta‐analysis, the median width of the CI of BR was 11.6% (interquartile range (IQR), 6.30%–18.5%). Incorporating this uncertainty in BR led to expansion of the RD CI by a median of 8 per 1,000 persons (IQR 2–24). This expansion increased in a linear fashion with BR imprecision and was more prominent in meta‐analyses with low BR. This study provides a web‐based tool to perform simple microsimulation and incorporate uncertainty in BR into the CI of RD.

Abstract Objective Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world 2-reviewer process. Materials and Methods A dataset of 10 trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n = 5) and held-out test sets (n = 17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the 2 LLMs were considered concordant if they were the same for a given variable. The discordant responses from each LLM were provided to the other LLM for cross-critique. Accuracy, ie, the total number of correct responses divided by the total number of responses, was computed to assess performance. Results In the prompt development set, 110 (96%) responses were concordant, achieving an accuracy of 0.99 against the gold standard. In the test set, 342 (87%) responses were concordant. The accuracy of the concordant responses was 0.94. The accuracy of the discordant responses was 0.41 for GPT-4-turbo and 0.50 for Claude-3-Opus. Of the 49 discordant responses, 25 (51%) became concordant after cross-critique, increasing accuracy to 0.76. Discussion Concordant responses by the LLMs are likely to be accurate. In instances of discordant responses, cross-critique can further increase the accuracy. Conclusion Large language models, when simulated in a collaborative, 2-reviewer workflow, can extract data with reasonable performance, enabling truly “living” systematic reviews.

ABSTRACTBackgroundMitral transcatheter edge‐to‐edge repair (TEER) is a minimally invasive therapy for severe mitral regurgitation (MR) in patients with high surgical risk. TEER results in a tissue bridge that decreases mitral valve area, potentially leading to elevated mean gradient. The clinical impact of elevated gradient on outcomes is unclear.MethodsThis systematic review and meta‐analysis was based on a comprehensive search of five databases. We included studies that evaluated post mitral TEER gradient on one of the following outcomes: all‐cause mortality, combined endpoints, symptoms, hospitalizations, and mitral valve re‐intervention. Meta‐analysis was conducted using the random‐effects model.ResultsOut of 6458 citations, 17 studies were included (2017−2024) reporting on 7748 patients. There was significant heterogeneity between studies related to the cutoff used to define elevated gradient, etiology of MR, and outcomes. Elevated mitral gradient post TEER (4, 4.5, 5 mmHg) was associated with worse combined endpoint (HR = 1.35, 95% CI 1.07−1.71; I2 64%), but not all‐cause mortality (HR = 1.25, 95% CI 0.97−1.59; I2 45%) or risk of hospitalization. For the combined endpoint, this association was mainly significant in patients who had mitral gradient assessed using discharge echocardiogram (HR = 1.39, 95% CI 1.07−1.81; I2 62%) for all patients and for those with degenerative MR.ConclusionDespite the heterogeneity between studies, the current analysis suggests that patients with elevated mitral gradient post TEER are at risk of worse clinical outcomes, particularly in patients who had mitral gradient assessed using discharge transthoracic echocardiogram.

Background Identifying a microorganism in patients with native vertebral osteomyelitis presents diagnostic challenges. Microorganism identification through culture-based methods is constrained by prolonged processing times and sensitivity limitations. Despite the availability of molecular diagnostic techniques for identifying microorganisms in native vertebral osteomyelitis, there is considerable variability in reported sensitivity and specificity across studies, leading to uncertainty in their clinical utility. Questions/purposes What are the sensitivity, specificity, and diagnostic odds ratios for 16S broad-range PCR followed by Sanger sequencing (16S) and metagenomic next-generation sequencing (NGS) for detecting bacteria in native vertebral osteomyelitis? Methods On June 29, 2023, we searched Cochrane, Embase, Medline, and Scopus for results from January 1970 to June 2023. Included studies involved adult patients with suspected native vertebral osteomyelitis undergoing molecular diagnostics—16S bacterial broad-range PCR followed by Sanger sequencing and shotgun or targeted metagenomic NGS—for bacteria detection. Studies involving nonnative vertebral osteomyelitis and cases of brucellar, tubercular, or fungal etiology were excluded. The reference standard for the diagnosis of native vertebral osteomyelitis was a composite clinical- and investigator-defined native vertebral osteomyelitis diagnosis. Diagnostic performance was assessed using a bivariate random-effects model. Risk of bias and diagnostic applicability were evaluated using the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. After a manual screening of 3403 studies, 10 studies (5 on 16S, 5 on NGS) were included in the present analysis, from which 391 patients were included from a total of 958 patients overall. Quality assessment via QUADAS-2 criteria showed moderate risk of bias and good applicability. Results 16S showed 78% (95% confidence interval [CI] 95% CI 31% to 96%) sensitivity and 94% (95% CI 73% to 99%) specificity, whereas NGS demonstrated 82% (95% CI 63% to 93%) sensitivity and 71% (95% CI 37% to 91%) specificity. In addition, the diagnostic ORs were 59 (95% CI 9 to 388) and 11 (95% CI 4 to 35) for 16S and NGS, respectively. Summary receiver operating characteristic curves showed high test performance for 16S (area under the curve for 16S 95% [95% CI 93% to 97%] and for NGS 89% [95% CI 86% to 92%]). Certainty in estimates was moderate because of sample size limitations. Conclusion This meta-analysis found moderate-to-high diagnostic performance of molecular methods on direct patient specimens for the diagnosis of native vertebral osteomyelitis. When used as a complementary test to microbiological analyses, a positive 16S result rules in the diagnosis of native vertebral osteomyelitis, while further studies are needed to understand the role of NGS in the diagnosis of native vertebral osteomyelitis. When available, these tests should be used in addition to conventional microbiology, especially in complex cases with extensively negative standard microbiological test results, to detect fastidious bacteria or to confirm the causative bacteria when their isolation and pathogenicity are unclear. A large sample size is needed in future research to understand the use of these techniques as standalone tests for diagnosis. Level of Evidence Level III, diagnostic study.

Abstract Objective To assess the association of aPL and diffuse alveolar haemorrhage (DAH) in patients with SLE by performing a systematic review and meta-analysis. Methods Multiple databases were systematically searched from inception to February 2024. Studies were eligible if they included patients with SLE (population), reported aPL status (exposure), and DAH (outcome). We pooled the estimates as odds ratio (OR) using fixed-effect models. We examined the association between aPL and DAH, as well as associations based on aPL subtypes or concomitant APS. Results Out of 454 screened studies, nine were included in meta-analysis, encompassing 7746 patients with SLE, of whom 2016 (26.0%) were aPL-positive and 163 (2.1%) had DAH. Patients with SLE and positive aPL (any) were more likely to develop DAH than aPL-negative patients (OR = 1.76, 95% CI 1.24–2.49; I2 = 0%). Patients with SLE and positive LA (OR = 1.76, 95% CI 1.06–2.93, I2 = 35%) or positive anticardiolipin IgG (OR = 1.62, 95% CI 1.13–2.34, I2 = 0%) had a higher likelihood of developing DAH compared with patients that were negative for these aPL. An APS diagnosis was associated with a 2.5-fold increased likelihood of DAH compared with subjects without APS (OR = 2.46, 95% CI 1.23–4.92, I2 = 0%). Positivity of anti-β2 glycoprotein I IgG was not significantly associated with DAH among patients with SLE (OR = 0.78, 95% CI 0.45–1.36, I2 = 0%). Conclusion In patients with SLE, aPL positivity increases the risk of DAH compared with aPL-negative patients, particularly in those positive for LA and anticardiolipin IgG.