Johnston, Bradley C.

When one initially targets the null effect and the point estimate falls close to the null, two challenges exist in rating certainty of evidence. First, when the point estimate is near the null and the data, therefore, suggests little or no effect, rating certainty in a benefit or harm is misleading. Second, since in general the narrower the confidence interval (CI) the more precise the estimate, if the CI is narrow, rating down for imprecision due simply to crossing the null is inappropriate. This paper addresses these issues and provides a solution: to revise the target of certainty rating from a non-zero effect to a little or no effect. This solution requires estimating a range in which the minimal important difference (MID) for benefit and an MID for harm might lie, and thus establishing a range that represents little or no effect. If GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) users are confident that the point estimate represents an effect less than the smallest plausible MID, they will revise their target and rate certainty to a little or no effect. If the entire CI falls within the range of little or no effect, they will not rate down for imprecision. Otherwise (if the CI includes an important effect), they will rate down. Using the solution provided in this paper GRADE users can make an optimal choice of the target of certainty rating.

Abstract Background Training with virtual reality (VR) simulation is increasingly used for health professions training to allow novices to practice in a learner-centered, risk-free environment. This review was performed to evaluate the effectiveness of VR simulation training in gastrointestinal endoscopy. Aims To determine whether virtual reality simulation training can supplement and/or replace early conventional endoscopy training in diagnostic endoscopy for health professions trainees with limited or no prior endoscopic experience. Methods We searched 17 databases from inception until October 18, 2023. We included randomised and quasi-randomised trials that compared VR simulation training with no training, conventional training, another form of simulation training, or an alternative method of VR training. We screened and abstracted data through Cochrane methodology. The primary outcome was composite score of competency. Secondary outcomes were procedure completion, procedure time, adverse events, patient discomfort, global rating of competency, and mucosal visualization. We calculated risk ratio for dichotomous outcomes and mean difference (MD) or standardised mean difference (SMD) for continuous outcomes with 95% CI. We used GRADE to assess the certainty of evidence. Results We included 20 trials (500 participants; 3975 endoscopic procedures). There was insufficient evidence to determine the effect of VR training on composite score of competency compared to no training or conventional training. VR training was advantageous over no training based for independent procedure completion (RR 1.62, 95% CI 1.15 to 2.26; moderate certainty evidence), overall rating of performance (mean difference [MD] 0.45, 95 %CI 0.15 - 0.75, very low certainty evidence), and mucosal visualization (MD 0.60, 95 %CI 0.20 - 1.00, very low certainty evidence). VR training resulted in fewer independent procedure completions compared to conventional training (RR = 0.45, 95 %CI 0.27 - 0.74, low certainty evidence). We found no differences between VR training and no training or conventional training for other outcomes. Based on qualitative analysis, we found no significant differences between VR training and other forms of simulation training. VR curricula based in educational theory provided benefit with respect to composite score of competency, compared with unstructured curricula. Based on qualitative analysis, we found no significant differences between VR training and other forms of simulation training. VR curricula based in educational theory provided benefit with respect to composite score of competency, compared with unstructured curricula. Conclusions VR simulation training in endoscopy can supplement conventional endoscopy training and provides benefit compared to no training. Risk of bias summary Funding Agencies None