Sewell, Justin L.

Abstract Background The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. Methods We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. Results Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). Conclusions We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.

Meaningful Equity, Diversity, and Inclusion (EDI) efforts may be stymied by concerns about whether proposed initiatives are performative or tokenistic. The purpose of this project was to analyze discussions by the Research in Medical Education (RIME) Program Planning committee about how best to recognize and support underrepresented in medicine (URiM) researchers in medical education to generate lessons learned that might inform local, national, and international actions to implement meaningful EDI initiatives. Ten RIME Program Planning Committee members and administrative staff participated in a focus group held virtually in August 2021. Focus group questions elicited opinions about “if and how” to establish a URiM research award. The focus group was recorded, transcribed, and thematically analyzed. Recognition of privilege, including who has it and who doesn’t, underpinned the focus group discussion, which revolved around 2 themes: (1) tensions between optics and semantics, and (2) potential unintended consequences of trying to level the medical education playing field. The overarching storyline threaded throughout the focus group discussion was intentionality. Focus group participants sought to avoid performativity by creating an award that would be meaningful to recipients and to career gatekeepers such as department chairs and promotion and tenure committees. Ultimately, participants decided to create an award that focused on exemplary Equity, Diversity, and Inclusion (EDI) scholarship, which was eventually named the “RIME URiM Research Award.” Difficult but productive conversations about EDI initiatives are necessary to advance underrepresented in medicine (URiM) scholarship. This transparent commentary may trigger further critical conversations.

In this Commentary, the authors seek to build on prior RIME commentaries by considering how researchers transition from worldviews, focal lengths, and research goals to research directions and methodological choices. The authors use the analogy of a hiker to illustrate how different researchers studying a similar phenomenon can choose among different research directions, which lead down different paths and offer different perspectives on a problem. Following the hiker analogy, the authors use the “Research Compass” to categorize the 15 research papers included in the 2020 Research in Medical Education supplement according to their research aim and corresponding methodological approach. The authors then discuss implications of the relative balance of these study types within this supplement and within health professions education research at large, emphasizing the critical importance of studying a topic from multiple vantage points to construct a richer and more nuanced understanding of health professions education challenges. The authors conclude by recognizing the challenges we face in the current era of COVID-19 and by calling health professions education researchers and practitioners to continue our collective efforts to improve learner education and patient care, as we together navigate the unfamiliar terrain of the present day.