Wu, Sarah

Abstract Problem In the United States, physician bias is exhibited early in medical training and contributes to systemic inequities within the field of medicine. A lack of diversity, equity, inclusion, and antiracism (DEI-AR) content within medical curricula drives critical gaps in knowledge and deficiencies when preparing medical students to serve patients of diverse backgrounds. At the Mayo Clinic Alix School of Medicine (MCASOM), student-led curricular reviews between 2017 to 2018 and 2020 to 2021 revealed opportunities to improve DEI-AR content within preclinical courses. Course directors expressed concern of limited expertise and time to enact effective changes. Approach The MCASOM DEI-AR teaching assistant (TA) program aims to curate a collaborative partnership between course directors and compensated student TAs to facilitate course enhancements responsive to the prior preclinical course review while centering standardized DEI-AR best practices. Outcomes As of January 2024, the program has engaged 14 TAs and partnered with 24 preclinical courses. Postcourse student evaluation responses were collected from 8 courses for 2021 to 2022 (before enhancements) and 2022 to 2023 (after enhancements). Student satisfaction with DEI-AR content is tracked through postcourse evaluations, with preliminary data demonstrating improvement after DEI-AR curricular integration (improvement of mean preenhancement and postenhancement scores of 3.81 to 4.05; t12 = 1.79, P = .21). Qualitative student comments were sorted into general categories of positive, negative, or neutral, showing a 6.25% median increase in positive perception of DEI. Next Steps Plans for the MCASOM DEI-AR TA program include application of quality improvement strategies to improve program processes and outcomes. Development of a centralized dashboard that integrates course enhancement progress and ongoing feedback from evaluations is anticipated to facilitate this effort. The program additionally aims to develop partnerships with clinical clerkships, which would allow for a more comprehensive enhancement of the overall medical education experience related to DEI-AR.

Abstract Introduction The Electronic Medical Record (EMR) has had a positive impact on healthcare but it has also increased administrative burden on physicians. One reason for this is the patient portal, which grants patients 24/7 access to ask their physician questions. Objective To characterize the patient message volume of urologists with a Men’s Health Practice by quantifying message volume per case, identifying procedures linked to high message volume, and examining demographics affecting patient portal use. We then assess the clinical utility of patient portal messages by examining their effect on Emergency Department (ED) utilization. Methods We queried the EMR of a health system spanning the southeast, midwest, and southwest using internal procedure and surgical codes for all office and operating room based urologic procedures from March 2017-November 2023. We identified procedures related to Men’s Health focusing on andrology, sexual medicine, and male reconstructive procedures. Patients identifying as women, those with multiple procedures within 6 months, and patients without research authorization were excluded. To identify factors associated with active portal users and message volume, we performed univariate and multivariable logistic and linear regression models. Multivariable models consisted of all statistically significant factors on univariate analysis. In models utilizing the procedure variable to compare message volume, vasectomy was chosen as the reference procedure due to it having the highest case volume. To assess the impact of active portal use (defined as sending at least 1 message within 6 months of their procedure) on risk of subsequent ED visit, we performed univariate Kaplan–Meier analysis and univariate and multivariable Cox Proportional Hazard models treating portal use as a time-dependent covariate. Results We examined 16,299 procedures which generated 49,422 messages within 6 months of their procedure (Table 1). Demographic data can be seen in table 2. On average, 3.04 ± 6.88 messages were generated after each procedure. The highest message generating procedures were testosterone Injection/pellets (5.83 ± 9.57), Itind (5.73 ± 14.18), and Transurethral Incision of the Prostate (5.19 ± 13.19). The lowest message generating procedures were vasectomy (1.30 ± 3.10), spermatocelectomy (1.67 ± 3.80), and circumcision (2.32 ± 5.26). The types of cases that generated the highest overall message burden were male reconstructive procedures (4.53 ± 8.57) and BPH procedures (4.07 ± 8.40) and. Slightly less than half of our cohort were active on the portal (45%) within 6 months of their procedure (Table 1). The procedures that were the most likely to have active portal users were microtese (OR 5.03 95% CI 2.38,10.67), testosterone injection/pellets (OR 4.86 95% CI 3.13, 7.54), and Itind (OR 3.02 95% CI 2.16, 4.24). On time to event analysis we found that portal usage decreased the risk of presenting to the Emergency Department (ED) by 15% within 90 days of their procedure [HR (95% CI) 0.65 0.85 (0.77, 0.93)] (Fig. 1). Conclusions Urologists practicing Men’s Health receive a high number of messages per procedure that they perform, however the use of the patient portal reduces ED utilization. Resources should be allocated to help urologists manage their patient messages as it lowers healthcare costs. Disclosure No.

Abstract Introduction Despite direct access to clinicians through the electronic health record, patients are increasingly turning to the internet for information related to their health, especially with sensitive urologic conditions such as Peyronie’s disease (PD). Large language model (LLM) chatbots are a form of artificial intelligence that rely on user prompts to mimic conversation, and they have shown remarkable capabilities. The conversational nature of these chatbots has the potential to answer patient questions related to PD; however, the accuracy, comprehensiveness, and readability of these LLMs related to PD remain unknown. Aims To assess the quality and readability of information generated from 4 LLMs with searches related to PD; to see if users could improve responses; and to assess the accuracy, completeness, and readability of responses to artificial preoperative patient questions sent through the electronic health record prior to undergoing PD surgery. Methods The National Institutes of Health’s frequently asked questions related to PD were entered into 4 LLMs, unprompted and prompted. The responses were evaluated for overall quality by the previously validated DISCERN questionnaire. Accuracy and completeness of LLM responses to 11 presurgical patient messages were evaluated with previously accepted Likert scales. All evaluations were performed by 3 independent reviewers in October 2023, and all reviews were repeated in April 2024. Descriptive statistics and analysis were performed. Results Without prompting, the quality of information was moderate across all LLMs but improved to high quality with prompting. LLMs were accurate and complete, with an average score of 5.5 of 6.0 (SD, 0.8) and 2.8 of 3.0 (SD, 0.4), respectively. The average Flesch-Kincaid reading level was grade 12.9 (SD, 2.1). Chatbots were unable to communicate at a grade 8 reading level when prompted, and their citations were appropriate only 42.5% of the time. Conclusion LLMs may become a valuable tool for patient education for PD, but they currently rely on clinical context and appropriate prompting by humans to be useful. Unfortunately, their prerequisite reading level remains higher than that of the average patient, and their citations cannot be trusted. However, given their increasing uptake and accessibility, patients and physicians should be educated on how to interact with these LLMs to elicit the most appropriate responses. In the future, LLMs may reduce burnout by helping physicians respond to patient messages.