Mustafa, Reem A

Abstract Introduction While certainty of evidence assessment is key to a rigorous and transparent systematic review, it is unknown how – and how frequently – it is assessed in systematic reviews. The objective of this study was to examine the prevalence and approaches used for certainty of evidence assessment in systematic reviews published in high‐impact medicine journals over the past 11 years. Methods A PubMed search and hand‐searching of relevant journal websites identified systematic reviews published between 24 January 2013 and 23 January 2024 in any of the ten highest‐impact journals in the General and Internal Medicine category of the Journal Citation Report. Two reviewers independently selected any systematic review related to health outcomes assessing certainty of evidence using any method. We extracted data related to review characteristics, certainty of evidence and risk of bias/methodological quality assessment frameworks, and reported consideration of certainty of evidence domains. Logistic regression examined year of publication to determine whether the prevalence of certainty of evidence assessment changed over time. Results Of 1,023 included reviews, 346 (33.8%) assessed certainty of evidence. Prevalence of certainty of evidence assessment increased over time (0.16 ± 0.2; p < .001). Most (89.3%) of reviews used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess certainty of evidence. Conclusion Only one in three systematic reviews published in the highest‐impact medical journals over the past 11 years assessed certainty of evidence, though prevalence increased over time. The use of specific domains within each certainty of evidence framework was not clearly described in all reviews.

When one initially targets the null effect and the point estimate falls close to the null, two challenges exist in rating certainty of evidence. First, when the point estimate is near the null and the data, therefore, suggests little or no effect, rating certainty in a benefit or harm is misleading. Second, since in general the narrower the confidence interval (CI) the more precise the estimate, if the CI is narrow, rating down for imprecision due simply to crossing the null is inappropriate. This paper addresses these issues and provides a solution: to revise the target of certainty rating from a non-zero effect to a little or no effect. This solution requires estimating a range in which the minimal important difference (MID) for benefit and an MID for harm might lie, and thus establishing a range that represents little or no effect. If GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) users are confident that the point estimate represents an effect less than the smallest plausible MID, they will revise their target and rate certainty to a little or no effect. If the entire CI falls within the range of little or no effect, they will not rate down for imprecision. Otherwise (if the CI includes an important effect), they will rate down. Using the solution provided in this paper GRADE users can make an optimal choice of the target of certainty rating.

Introduction: For cardiac transthyretin (ATTR) amyloidosis, bone scintigraphy has emerged as a noninvasive diagnostic tool. The use of different tracers, namely Pyrophosphate (PYP), 3,3-diphosphono-1,2-propanodicarboxylic acid (DPD), HydroxymethyleneDiphosphonate (HMDP), allow for the detection of amyloid deposits in the heart. The aim of this systematic review is to evaluate the diagnostic test accuracy of bone scintigraphy in ATTR and light chain (AL) amyloidosis. Methods: As part of the ASH AL Amyloidosis Diagnostic Guideline, we performed systematic reviews for 7 prioritized PICO questions related to the screening, diagnosis and organ involvement evaluation of AL amyloidosis. Electronic searches were conducted on PubMed, Embase, and the Cochrane Central Register of Controlled Trials from inception till January 2024. Two reviewers independently and in duplicate performed title and abstract screening and full text article screening on LASER AI, with conflicts resolved by a third reviewer. Studies were eligible if they included patients that performed both a bone scintigraphy scan and an endomyocardial biopsy for suspicion of ATTR amyloidosis. Test positivity for the visual score was defined to be ≥2, for the heart over contralateral (H/CL) ratio it was &gt; 1.5 at one hour and &gt;1.3 at 3 hours. Diagnostic test accuracy measures of interest were sensitivity, specificity, positive and negative predictive value. Statistical analysis was performed on STATA 18.0. Grading of Recommendations Assessment, Development and Evaluation (GRADE) was used to assess the certainty of evidence. Results: After deduplication, we retrieved 29,237 studies from electronic databases. 12 studies with 1127 patients reported on visual score in patients suspected to have ATTR amyloidosis. The pooled sensitivity for visual score was 96% (95% CI 92.0-98.0) while specificity was 90% (95% CI 84.0-95.0) (Certainty of evidence (COE): Moderate ⨁⨁⨁◯). 3 studies reported on ATTR visual score in combination with evaluating for monoclonal gammopathy, the pooled sensitivity was decreased to 93% (95% CI 65-99) and while specificity increased to 99% (95% CI 80-100) (COE: Moderate ⨁⨁⨁◯). H/CL was evaluated in 5 studies with 189 patients; pooled sensitivity was 97% (95% CI 89.0-99.0) and specificity was 88% (95% CI 64-97) (COE: Low ⨁⨁◯◯). We also performed subgroup analyses for the diagnostic test accuracy of visual score for the different tracers (PYP, DPD, HMDP) in patients suspected to have ATTR amyloidosis. DPD was found to have the highest sensitivity which was found to be 97% (95% CI 91.0-99.0) and a specificity of 95 (95% CI 86.0-98.0) (4 studies, n = 297 COE: Moderate ⨁⨁⨁◯). The sensitivity and specificity for PYP was 95% (95% CI 87.0-98.0) and 90% (95% CI 79.0-96.0) (7 studies, n = 377 COE: Moderate ⨁⨁⨁◯). 1 study reported on HMDP with a sensitivity and specificity of 79% (95% CI 49.0-95.0) and 100% (95% CI 59.0-100.0) (COE: Low ⨁⨁◯◯). Conclusion: Bone scintigraphy has a high sensitivity and specificity for the diagnosis of ATTR amyloidosis, particularly in patients without a monoclonal gammopathy.

Introduction: Although endomyocardial biopsy is the gold standard, echocardiography, and Cardiac Magnetic Resonance (CMR) play a pivotal role in the diagnosis of cardiac amyloidosis. In this systematic review, we aim to estimate the diagnostic test accuracy (DTA) of CMR and echocardiography for cardiac amyloidosis. Methods: As part of the American Society of Hematology guidelines on the diagnosis of Amyloidosis, we searched PubMed, Embase, and the Cochrane Central Register of Controlled Trials from inception till January 2024 for relevant studies. Two reviewers independently performed title and abstract screening and full-text article screening on LASER Al and extracted relevant data using a piloted Excel sheet. We performed statistical pooling using Stata 18.0 software. We evaluated risk of bias using Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) and used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to assess the certainty of the evidence. We report diagnostic test accuracy as sensitivity (95% confidence interval) and specificity (95% confidence interval). Results: After screening 29,237, we included 9 studies assessing DTA of late gadolinium enhancement (LGE) on cardiac magnetic resonance (CMR) using an endomyocardial biopsy (EMB) as a reference test in patients with amyloidosis (light chain and ATTR). The majority of studies didn't report on the type of amyloidosis where only 2 out of 9 were in patients with light chain amyloidosis. The pooled estimates for sensitivity and specificity of LGE are 0.95 (0.88. 0.98), and 0.81 (0.60, 0.92), respectively. We did a sensitivity analysis and included only LGE suggestive for amyloidosis which didn't change sensitivity but led to an increase in specificity for amyloidosis to 0.87 (0.76, 0.94), respectively. The pooled estimates were based on moderate certainty in the evidence. For 2-dimensional echocardiography, we addressed DTA for different echocardiographic findings and we did not limit the reference to EMB. A panel of experts judged whether the reference was acceptable or not. Acceptable reference was EMB alone or a combination of extracardiac biopsy plus cardiac imaging (CMR or echocardiography) with or without positive cardiac biomarkers (troponin or BNP or NT-proBNP). Six studies (n=2063) addressed interventricular septum thickness (IVS), sensitivity was 0.77 (0.69, 0.84) and specificity was 0.71 (0.60, 0.81). 4 out of 6 studies used 12 mm as a cutoff while the other 2 studies used 13 mm as a cutoff. Only 2 studies (n=1667) assessed the DTA of posterior wall thickness (PWT), with sensitivity ranging from 0.76 to 1.00, and specificity ranging from 0.84 to 0.89. Six studies (n=438) addressed diastolic dysfunction (grade 2,3), with a sensitivity of 0.71 (0.40, 0.90), and specificity of 0.75 (0.64, 0.84). 7 studies (n=472) reported DTA of only diastolic dysfunction grade 3. When compared to diastolic dysfunction (grade 2,3, sensitivity decreases to 0.42 (0 28. 0.58) and specificity increases to 0.89 (0.83. 0.94). For E/A ratio, only 2 studies (n=127) were included. Sensitivity ranged from 0.45 to 0.65, and specificity ranged from 0.85 to 0.98. Four studies (n=1596) yielded a sensitivity of 0.86 (0.65, 0.95) and specificity of 0.76 (0.55, 0.89) of left ventricular global longitudinal strain (GLS). All except one used -17% as a cutoff for GLS. For apical sparing on GLS, pooled estimates from 8 studies (n=2253) led to a sensitivity of 0.72 (0.64, 0.78) and specificity of 0.78 (0.64, 0.88). There was a variation of cutoff across studies. The certainty of evidence for echocardiographic findings was low to moderate. Conclusion: CMR is a more accurate test with better performance compared to echocardiography.

Introduction: Light-chain (AL) amyloidosis is caused by the secretion of immunoglobulin light chains by monoclonal plasma cells. The increase in the levels of these monoclonal proteins may proceed the development of any clinical symptoms, and their identification in the serum and urine remains an integral part of establishing a diagnosis of AL amyloidosis. This systematic review aims to evaluate the diagnostic test accuracy of serum immunofixation (SIFE), urine immunofixation (UIFE), and free light chain (FLC) ratio for the diagnosis of AL amyloidosis. Methods: As part of the ASH AL Amyloidosis Diagnostic Guideline, we performed systematic reviews for 7 prioritized PICO questions related to the screening, diagnosis and organ involvement evaluation of AL amyloidosis. Electronic searches were conducted on PubMed, Embase, and the Cochrane Central Register of Controlled Trials from inception till January 2024. Two reviewers independently and in duplicate performed title and abstract screening and full text article screening on LASER AI, with conflicts resolved by a third reviewer. Eligible studies were those that included patients who performed one of the three tests (SIFE, UIFE, and κ/λ FLC) and had their AL amyloidosis diagnosis confirmed via a biopsy. Diagnostic test accuracy measures of interest were sensitivity, specificity, positive and negative predictive value. Statistical analysis was performed on OpenMeta[Analyst]. Grading of Recommendations Assessment, Development and Evaluation (GRADE) was used to assess the certainty of evidence. Results: After deduplication, we retrieved 29,237 studies from electronic databases. 31 studies with 3973 patients reported on the use of serum immunofixation (SIFE) with a pooled sensitivity of 72.4% (95% CI 68.1%-76.4%). 27 studies with 2855 patients reported on the use of urine immunofixation UIFE for the diagnosis of AL amyloidosis with a pooled sensitivity of 78.6% (95% CI 74.6%-82%). 29 studies with 5305 patients reported on the use of κ/λ FLC ratio amyloidosis with a pooled sensitivity of 79.9% (95% CI 73.7%-84.9%). 24 studies reported on the use of both SIFE and UIFE with a sample size of 3751 patients and a pooled sensitivity of 90% (95% CI 85.3%-93.3%). 3 studies reported on the use SIFE and κ/λ FLC ratio with 321 patients and a pooled sensitivity of 97.7% (95% CI 95.6%-99.9%). Finally, we identified 10 studies that performed SIFE,UIFE, and κ/λ FLC ratio concurrently with 1926 patients and a pooled sensitivity of 99% (95% CI 97%-99.7%) of all three tests and considering a result abnormal if any of the components had a positive result. We also identified 5 studies with 957 patients reporting on the concurrent use of SIFE, UIFE, and κ/λ FLC ratio for the diagnosis AL amyloidosis and differentiation of AL from ATTR amyloidosis in a population of patients with suspected cardiac amyloid involvement, the pooled sensitivity was found to be 99%(95% CI 99%-100%) and with a pooled specificity of 77%(95% CI 75%-80%). Certainty of evidence moderate for all outcomes. Conclusion: The combined use of SIFE, UIFE, and κ/λ FLC ratio has a high sensitivity for the diagnosis of AL amyloidosis, and the tests remain a valuable initial screening of AL amyloidosis and differentiation from ATTR amyloidosis.

Introduction: In patients with systemic light-chain amyloidosis, cardiac involvement is common. Cardiac involvement (presence and severity) is the most important predictor of mortality in patients with AL amyloidosis. Endomyocardial biopsy remains the gold standard, but it is a costly and invasive test. Consequently, cardiac biomarkers are used for screening for cardiac involvement and have significant prognostic value.In this systematic review, we aim to estimate the diagnostic test accuracy (DTA) of Nt-proBNP, BNP, and high sensitivity troponin for cardiac amyloidosis. Methods: As part of the American Society of Hematology guidelines on the diagnosis of Amyloidosis, we searched PubMed, Embase, and the Cochrane Central Register of Controlled Trials from inception till January 2024 for relevant studies. Two reviewers independently performed title and abstract screening and full-text article screening on LASER Al and extracted relevant data using piloted Excel sheets. The panel judged whether the reference test is acceptable or not. Acceptable reference test is EMB alone or a combination of extracardiac biopsy plus cardiac imaging (CMR or echocardiography) with or without positive cardiac biomarkers (troponin or BNP or NT-proBNP). For statistical analysis, we only included studies reporting on both the sensitivity and specificity of the cardiac biomarker. We used Stata 18.0 software for meta-analysis. We evaluated risk of bias using Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) and used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to assess the certainty of the evidence. We report diagnostic test accuracy as sensitivity (95% confidence interval) and specificity (95% confidence interval). Results: After screening 29,237 studies, a total of 10 studies addressed DTA of Nt-proBNP for cardiac amyloidosis. These studies used a wide variety of cutoffs for diagnosis of cardiac involvement, ranging from 150 to 5000 ng/L and a cutoff of 332 ng/L was the most studied and reported. The pooled estimates for sensitivity and specificity of Nt-proBNP using 332 as cutoffs are 0.99 ( 0.97, 1) and 0.53 (0.3, 0.75), respectively. The pooled estimates were based on 4 studies (n=645) with moderate certainty in the evidence. For BNP, DTA was assessed in 3 studies (n= 257). Each study used a different cutoff for BNP (73, 100, and 412 ng/L). The sensitivity and specificity of BNP in each study were comparable and not significantly affected by the cutoff variation. Pooled sensitivity and specificity across studies, were 0.87 (0.80, 0.92), and 0.79 (0.71, 0.85), respectively. The certainty in evidence for the DTA of BNP was low. Eight studies reported on DTA of high sensitivity troponin, with cutoffs ranging from 14 to 86 ng/L. Meta-analysis of studies reporting on cutoff 14 ng/L yielded sensitivity of 0.94 (0.87, 0.97) and specificity of 0.11 (0.08, 0.16) with moderate certainty evidence. Increasing the cutoff to 35 ng/l led to a decrease in sensitivity to 0.82 (0.66, 0.92) and an increase in specificity to 0.86 (0.74, 0.93) with low certainty evidence. Conclusion: Cardiac biomarkers are highly sensitive, valuable initial tests for increasing suspicion of cardiac involvement in patients with AL amyloidosis. However, they lack specificity and, on their own, are insufficient for a definitive diagnosis of cardiac involvement. These biomarkers may be used in conjunction with other diagnostic tests to accurately identify and assess the extent of cardiac involvement.

Introduction: Tissue biopsies are needed to establish a diagnosis of systemic AL amyloidosis. Biopsy can be obtained from an involved organ with suspected amyloid deposition (target biopsy) or from sites in which amyloid commonly deposits (surrogate biopsy). The choice of biopsy site depends on institutional and physician preference. Two commonly used sites for surrogate biopsies are abdominal fat pad and bone marrow. The present systematic review and meta-analysis aims to evaluate the diagnostic test accuracy of fat pad and bone marrow biopsies as surrogate biopsy sites in the diagnosis of AL amyloidosis. Methods: As part of the ASH AL Amyloidosis Diagnostic Guideline, we performed systematic reviews for seven prioritized PICO questions related to the screening, diagnosis and organ involvement evaluation of AL amyloidosis. We conducted electronic searches on PubMed, Embase, and the Cochrane Central Register of Controlled Trials from inception until January 2024. Two reviewers independently and in duplicate performed title and abstract screening and full text article screening on LASER AI, with conflicts resolved by a third reviewer. Studies were eligible if they included patients that had a fat pad sample or bone marrow biopsy and a target organ biopsy for suspicion of AL amyloidosis allowing for the calculation of diagnostic test accuracy measures. Diagnostic test accuracy measures of interest were sensitivity, specificity, positive and negative predictive value. Statistical analysis was performed on OpenMeta[Analyst]. Grading of Recommendations Assessment, Development and Evaluation (GRADE) was used to assess the certainty of evidence. Results: After deduplication, we retrieved 29,237 studies.. 20 studies with 2955 patients reporting on bone marrow biopsy, 25 with 4702 patients on fat pad sampling were included. The pooled sensitivity for bone marrow biopsy was 55.1% (95% CI 45.8-64.0, I2 = 94%) (Certainty of evidence: Low ⨁⨁◯◯), For fat pad sampling sensitivity was 76.6% (95% CI 72.1-80.8, I2 = 84.2%) (Certainty of evidence: Moderate ⨁⨁⨁◯). Among the included studies, 1 evaluated the sensitivity of fat pad sampling and bone marrow biopsy combined and found it to be 89%. No included studies evaluated specificity for bone marrow biopsy or fat pad sample. Conclusion: Combination of fat pad and bone marrow biopsy together significantly improve sensitivity for amyloid detection compared to either fat pad sampling or bone marrow biopsy alone.

Abstract Background The Grading Recommendations, Assessment, Development, and Evaluations (GRADE) framework is widely applied in clinical guidelines to facilitate transparent evidence evaluation. While developing Infectious Diseases Society of America (IDSA) guidelines on the management of patients with coronavirus disease 2019 (COVID-19), panel members suggested developing and implementing a visual aid to enable quicker identification of key information by providers at bedside seeking guidance. Methods We conducted a mixed-methods study evaluating the usability of a newly designed infographic/icon using a survey and focus groups. The survey incorporated a simulated COVID-19 IDSA guideline with and without the icon, followed by comprehension questions. Focus group discussions provided qualitative feedback on the GRADE methodology and icon usability. Results The survey was returned by 289 health care providers. There was no statistical difference in the correct response rates between icon-aided and non-icon-aided guideline questions (McNemar's chi-square test, P &gt; .1 for both questions). Interactions with the icon notably increased the time taken and number of clicks required to respond to the first question (Wilcoxon signed-rank test, P &lt; .01). In contrast, response time did not differ between versions for the second question (P = .38). Most subjects (85%) indicated that the icon improved the readability of the guidelines. A focus group follow-up suggested alternative designs for the icon. Conclusions This study highlights the promise of iconography in clinical guidelines, although the specific icons tested did not measurably improve usability metrics. Future research should focus on icon design and testing within a formal usability framework, considering the impact of GRADE language on user experience.

Abstract Background The role of serologic testing for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has evolved during the pandemic as seroprevalence in global populations has increased. The Infectious Diseases Society of America convened an expert panel to perform a systematic review of the coronavirus disease 2019 (COVID-19) serology literature and construct updated best practice guidance related to SARS-CoV-2 serologic testing. This guideline is an update to the fourth in a series of rapid, frequently updated COVID-19 guidelines developed by the Infectious Diseases Society of America. Objective To develop evidence-based recommendations and identify unmet research needs pertaining to the use of anti–SARS-CoV-2 antibody tests for diagnosis, decisions related to vaccination and administration of monoclonal antibodies or convalescent plasma in immunocompromised patients, and identification of a serologic correlate of immunity. Methods A multidisciplinary panel of infectious diseases clinicians, clinical microbiologists, and experts in systematic literature reviewed, identified, and prioritized clinical questions related to the use of SARS-CoV-2 serologic tests. Grading of Recommendations Assessment, Development and Evaluation methodology was used to assess the certainty of evidence and make testing recommendations. Results The panel recommends against serologic testing to diagnose SARS-CoV-2 infection in the first 2 weeks after symptom onset (strong recommendations, low certainty of evidence). Serologic testing should not be used to provide evidence of COVID-19 in symptomatic patients with a high clinical suspicion and repeatedly negative nucleic acid amplification test results (strong recommendation, very low certainty of evidence). Serologic testing may assist with the diagnosis of multisystem inflammatory syndrome in children (strong recommendation, very low certainty of evidence). To seek evidence for prior SARS-CoV-2 infection, the panel suggests testing for immunoglobulin G (IgG), IgG/IgM, or total antibodies to nucleocapsid protein 3 to 5 weeks after symptom onset (conditional recommendation, low certainty of evidence). In individuals with previous SARS-CoV-2 infection or vaccination, the panel suggests against routine serologic testing given no demonstrated benefit to improving patient outcomes (conditional recommendation, very low certainty of evidence). The panel acknowledges further that a negative spike antibody test may be a useful metric to identify immunocompromised patients who are candidates for immune therapy. Conclusions The high seroprevalence of antibodies against SARS-CoV-2 worldwide limits the utility of detecting anti-SARS CoV-2 antibody. The certainty of available evidence supporting the use of serology for diagnosis was graded as very low to low. Future studies should use serologic assays calibrated to a common reference standard.

Abstract Accurate molecular diagnostic tests are necessary for confirming a diagnosis of coronavirus disease 2019 (COVID-19) and for identifying asymptomatic carriage of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). The number of available SARS-CoV-2 nucleic acid detection tests continues to increase as does the COVID-19 diagnostic literature. Thus, the Infectious Diseases Society of America (IDSA) developed an evidence-based diagnostic guideline to assist clinicians, clinical laboratorians, patients, and policymakers in decisions related to the optimal use of SARS-CoV-2 nucleic acid amplification tests. In addition, we provide a conceptual framework for understanding molecular diagnostic test performance, discuss nuances of test result interpretation in a variety of practice settings, and highlight important unmet research needs related to COVID-19 diagnostic testing. IDSA convened a multidisciplinary panel of infectious diseases clinicians, clinical microbiologists, and experts in systematic literature review to identify and prioritize clinical questions and outcomes related to the use of SARS-CoV-2 molecular diagnostics. Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology was used to assess the certainty of evidence and make testing recommendations. The panel agreed on 12 diagnostic recommendations. Access to accurate SARS-CoV-2 nucleic acid testing is critical for patient care, hospital infection prevention, and the public health response to COVID-19 infection. Information on the clinical performance of available tests continues to grow, but the quality of evidence of the current literature to support this updated molecular diagnostic guideline remains moderate to very low. Recognizing these limitations, the IDSA panel weighed available diagnostic evidence and recommends nucleic acid testing for all symptomatic individuals suspected of having COVID-19. In addition, testing is suggested for asymptomatic individuals with known or suspected contact with a COVID-19 case when the results will impact isolation/quarantine/personal protective equipment (PPE) usage decisions. Evidence in support of rapid testing and testing of upper respiratory specimens other than nasopharyngeal swabs, which offer logistical advantages, is sufficient to warrant conditional recommendations in favor of these approaches.