Citations

AbstractIntroduction/AimsGraduate medical education programs must ensure residents and fellows acquire skills needed for independent practice. Workplace‐based observational assessments are informative but can be time‐ and resource‐intensive. In this study we sought to gather “relations‐to‐other‐variables” validity evidence for scores generated by the Electromyography Direct Observation Tool (EMG‐DOT) to inform its use as a measure of electrodiagnostic skill acquisition.MethodsScores on multiple assessments were compiled by trainees during Clinical Neurophysiology and Electromyography rotations at a large US academic medical center. Relationships between workplace‐based EMG‐DOT scores (n = 298) and scores on a prerequisite simulated patient exercise, patient experience surveys (n = 199), end‐of‐rotation evaluations (n = 301), and an American Association of Neuromuscular & Electrodiagnostic Medicine (AANEM) self‐assessment examination were assessed using Pearson correlations.ResultsAmong 23 trainees, EMG‐DOT scores assigned by physician raters correlated positively with end‐of‐rotation evaluations (r = 0.63, P = .001), but EMG‐DOT scores assigned by technician raters did not (r = 0.10, P = .663). When physician and technician ratings were combined, higher EMG‐DOT scores correlated with better patient experience survey scores (r = 0.42, P = .047), but not with simulated patient or AANEM self‐assessment examination scores.DiscussionEnd‐of‐rotation evaluations can provide valid assessments of trainee performance when completed by individuals with ample opportunities to directly observe trainees. Inclusion of observational assessments by technicians and patients provides a more comprehensive view of trainee performance. Workplace‐ and classroom‐based assessments provide complementary information about trainee performance, reflecting underlying differences in types of skills measured.

Abstract Purpose To examine associations of social support and social isolation with burnout, program satisfaction, and organization satisfaction among a large population of U.S. residents and fellows and to identify correlates of social support and social isolation. Method All residents and fellows enrolled in graduate medical education programs at Mayo Clinic sites were surveyed in February 2019. Survey items measured social support (emotional and tangible), social isolation, burnout, program satisfaction, and organization satisfaction. Factors of potential relevance to social support were collected (via the survey, institutional administrative records, and interviews with program coordinators and/or program directors) and categorized as individual, interpersonal, program, or work-related factors (duty hours, call burden, elective time, vacation days used before survey administration, required away rotations, etc.). Multivariable regression analyses were conducted to examine relationships between variables. Results Of 1,146 residents surveyed, 762 (66%) from 58 programs responded. In adjusted models, higher emotional and tangible support were associated with lower odds of burnout and higher odds of program and organization satisfaction, while higher social isolation scores were associated with higher odds of burnout and lower odds of program satisfaction and organization satisfaction. Independent predictors of social support and/or social isolation included age, gender, relationship status, parental status, postgraduate year, site, ratings of the program leadership team, ratings of faculty relationships and faculty professional behaviors, satisfaction with autonomy, and vacation days used before survey administration. Conclusions This study demonstrates that social support and social isolation are strongly related to burnout and satisfaction among residents and fellows. Personal and professional relationships, satisfaction with autonomy, and vacation days are independently associated with social support and/or social isolation, whereas most program and work-related factors are not. Additional studies are needed to determine if social support interventions targeting these factors can improve well-being and enhance satisfaction with training.

Abstract Problem Faculty at academic health centers have many competing demands on their time, leading to high work stress, burnout, and limited capacity to meaningfully improve their teaching, evaluation, feedback, and other education-related skills. Faculty development provides a useful mechanism to assist faculty in enhancing their knowledge and skills in these areas, but engaging faculty can be a challenge. Approach To promote engagement, the authors developed a multipronged, pragmatic approach to faculty development. They created: (1) brief videos leveraging micro-learning strategies; (2) prepackaged workshops for use during existing faculty meetings; (3) a newsletter to raise awareness of faculty development opportunities; (4) a searchable, web-based catalog to facilitate rapid retrieval of faculty development content; and (5) an academy to acknowledge engagement of individual faculty members, provide certificates, and promote a culture that prioritizes our education mission. Outcomes Since they launched the new approach in 2017, they have developed 41 microlearning videos, 15 prepackaged workshops, and 24 issues of the newsletter. Between January 2017 and May 2021, the videos generated more than 150,055 views; the workshops were downloaded 2,850 times; and the issues of the newsletter, emailed bimonthly to 3,500 members of the faculty, had an open rate that increased from 30% in 2017 to 70% in 2021. The Academy of Educational Excellence, which was launched in 2018, grew to more than 490 members. Preliminary feedback suggests faculty and education leaders are highly satisfied with the faculty development resources and approaches to engagement. Next Steps Next steps include obtaining more user satisfaction data and evaluating whether education-related knowledge and skills have improved among faculty participants.

Purpose To evaluate the relationship between help-seeking concerns and attitudes and burnout among residents. Method In 2019, all residents across the 4 Mayo Clinic sites were surveyed. The survey included 2 items from the Maslach Burnout Inventory, an item from the National Comorbidity Survey Replication about likelihood of seeking professional help for a serious emotional problem, and items developed to explore residents’ help-seeking behaviors and concerns. Multivariable logistic regression was conducted for each outcome variable and included age, gender, specialty, postgraduate year, site, and burnout. Results Of the 1,146 residents to whom surveys were sent, 762 (66.5%) responded. Nearly half (342/747, 45.8%) were concerned about negative consequence to their career if they went on medical leave, and one-third (247/753, 32.8%) were reluctant to seek professional help for a serious emotional concern. Of the 437 residents who had never attended a personal health appointment during scheduled work, 34.6% (151) thought it would be difficult to tell a supervising physician they needed to miss work due to a scheduled appointment. On multivariable analysis, burnout was independently associated with reporting it would be difficult to tell a supervising physician of a need to attend an appointment (odds ratio [OR] 2.32; 95% confidence interval [CI] 1.46, 3.67; P < .001), being concerned about negative consequence to their career if they went on medical leave (OR 2.09; 95% CI 1.49, 2.93; P < .001), and reluctance to seek professional care for a serious emotional problem (OR 1.65; 95% CI 1.17, 2.34; P = .004). Conclusions Barriers to self-care and help-seeking are common among residents and may be worse among those with burnout. Strategies to reduce stigma and promote a culture of well-being are needed.

In this Commentary, the authors seek to build on prior RIME commentaries by considering how researchers transition from worldviews, focal lengths, and research goals to research directions and methodological choices. The authors use the analogy of a hiker to illustrate how different researchers studying a similar phenomenon can choose among different research directions, which lead down different paths and offer different perspectives on a problem. Following the hiker analogy, the authors use the “Research Compass” to categorize the 15 research papers included in the 2020 Research in Medical Education supplement according to their research aim and corresponding methodological approach. The authors then discuss implications of the relative balance of these study types within this supplement and within health professions education research at large, emphasizing the critical importance of studying a topic from multiple vantage points to construct a richer and more nuanced understanding of health professions education challenges. The authors conclude by recognizing the challenges we face in the current era of COVID-19 and by calling health professions education researchers and practitioners to continue our collective efforts to improve learner education and patient care, as we together navigate the unfamiliar terrain of the present day.

Purpose To explore the relationship between residents’ perceptions of residency program leadership team behaviors and resident burnout and satisfaction. Method In February 2019, the authors surveyed all residents across the 77 graduate medical education training programs at Mayo Clinic’s multiple sites. Survey items measured residents’ perceptions of program director and associate program director behaviors (using a composite residency program leadership team score), resident burnout, and resident satisfaction with the program and organization. Multivariable logistic regression was performed to evaluate relationships between these variables at the individual resident (adjusting for age, sex, postgraduate training year, program location, and specialty) and program (including only programs with at least 5 respondents) levels. Results Of the 1,146 residents surveyed, 762 (66.5%) responded. At the individual resident level, higher composite leadership team scores were associated with lower emotional exhaustion and depersonalization and higher overall satisfaction with the residency program and organization (all P < .001). In adjusted logistic regression models, each 1-point gain in composite leadership team score was associated with 9% lower odds of burnout, 20% higher odds of program satisfaction, and 19% higher odds of satisfaction with the organization (all P < .001). At the residency program level, higher mean composite leadership team scores were associated with a lower rate of burnout (r = −0.35, P = .03) and higher program and organization satisfaction (r = 0.67 and 0.74, respectively, both P < .001). Conclusions The behaviors of residency program leadership teams influence residents’ burnout and satisfaction. Additional studies are needed to determine if leadership training results in improved resident well-being and satisfaction.

Purpose To examine validity evidence for a standardized patient scenario assessing medical students’ ability to promote value using patient-centered communication (in response to a patient requesting an unnecessary test) and to explore the potential effect of various implementation and curricular factors on student scores. Method Third-year medical students (N = 516) from 5 U.S. MD-granting medical schools completed the communication scenario between 2014 and 2017 as part of a larger objective structured clinical examination (OSCE). Centralized raters assessed performance using an 11-item checklist. The authors collected multiple sources of validity evidence. Results The mean checklist score was 0.85 (standard deviation 0.09). Interrater reliability for checklist scores was excellent (0.87, 95% confidence interval = 0.78–0.93). Generalizability and Phi-coefficients were, respectively, 0.65 and 0.57. Scores decreased as the number of OSCE stations increased ( r = −0.15, P = .001) and increased when they were used for summative purposes ( r = 0.26, P < .001). Scores were not associated with curricular time devoted to high-value care ( r = 0.02, P = .67) and decreased when more clerkships were completed before the assessment ( r = −0.12, P = .006). Conclusions This multisite study provides validity evidence supporting the use of scenario scores to assess the ability of medical students to promote value in clinical encounters using patient-centered communication. Findings illuminate the potential effect of OSCE structure and purpose on student performance and suggest clerkship learning experiences may not reinforce what students are taught in the formal curriculum regarding high-value care. Devoting more time to the topic appears insufficient to counteract this erosion.