Gordon, Joel E

Abstract Background The Grading Recommendations, Assessment, Development, and Evaluations (GRADE) framework is widely applied in clinical guidelines to facilitate transparent evidence evaluation. While developing Infectious Diseases Society of America (IDSA) guidelines on the management of patients with coronavirus disease 2019 (COVID-19), panel members suggested developing and implementing a visual aid to enable quicker identification of key information by providers at bedside seeking guidance. Methods We conducted a mixed-methods study evaluating the usability of a newly designed infographic/icon using a survey and focus groups. The survey incorporated a simulated COVID-19 IDSA guideline with and without the icon, followed by comprehension questions. Focus group discussions provided qualitative feedback on the GRADE methodology and icon usability. Results The survey was returned by 289 health care providers. There was no statistical difference in the correct response rates between icon-aided and non-icon-aided guideline questions (McNemar's chi-square test, P > .1 for both questions). Interactions with the icon notably increased the time taken and number of clicks required to respond to the first question (Wilcoxon signed-rank test, P < .01). In contrast, response time did not differ between versions for the second question (P = .38). Most subjects (85%) indicated that the icon improved the readability of the guidelines. A focus group follow-up suggested alternative designs for the icon. Conclusions This study highlights the promise of iconography in clinical guidelines, although the specific icons tested did not measurably improve usability metrics. Future research should focus on icon design and testing within a formal usability framework, considering the impact of GRADE language on user experience.

Abstract Objective After a new electronic health record (EHR) was implemented at Mayo Clinic, a training program called reBoot Camp was created to enhance ongoing education in response to needs identified by physician leaders. Materials and Methods A reBoot camp focused on EHR topics pertinent to ambulatory care was offered from April 2018 through June 2020. There were 37 2-day sessions and 43 1-day sessions, with 673 unique participants. To evaluate outcomes of the reBoot camp, we used survey data to study baseline, immediate, and long-term perceptions of program satisfaction and self-assessed skills with the EHR. The study was conducted among practitioners at a large ambulatory practice network based in several states. Data were collected from April 2018 through January 2021. We analyzed automatically collected metadata and scores that evaluated the amount of personalization and proficiency of use. Results Confidence in skills increased by 13.5 points for general EHR use and was significant in 5 subdomains of use (13–18 point improvement). This degree of user confidence was maintained at the 6-month reassessment. The outcomes of configuration and proficiency scores also improved significantly. Discussion Ongoing education regarding EHR tools is necessary to support continued use of technology. This study was novel because of the amount and breadth of data collected, diversity of user participation, and validation that improvements were maintained over time. Conclusions Participating in a reBoot camp significantly improved user confidence in each domain of the EHR and demonstrated use of best-practice tools. Users maintained gains at the 6-month evaluation phase.