Paulson, Deirdre

Background and Objectives: We developed the Mayo Outpatient Precepting Evaluation Tool (MOPET) to assess and improve clinical teaching for family medicine residents. Given the scheduling challenges of peer evaluation, we sought to improve the MOPET’s usefulness by evaluating the reliability of self- and resident-reported teaching behaviors during clinical precepting compared to peer observation. Methods: Forty-eight half-day sessions in a family medicine residency clinic were assessed with parallel observer, preceptor, and resident MOPET forms. Interrater reliability was calculated using the κ (kappa) statistic. Results: Residents reported that preceptors used 14 of 20 teaching behaviors more often than observers. Observers and preceptors significantly agreed on the use of five teaching behaviors, while residents agreed with observers on two teaching behaviors. Conclusions: Although self-assessment and resident-based feedback of preceptor teaching behavior may improve practicality, they are less reliable compared to direct peer observation with the validated MOPET.

Background and Objectives: There are few trainings for primary care physicians (PCPs) on how to navigate a psychiatric crisis (such as suicidal ideation, homicidal ideation, psychosis, and/or intoxication that impairs decision making capacity) in the outpatient setting. We previously published an approach to fill this gap and now report the results of our efforts to improve the curriculum and assess if confidence was maintained over time. Methods: Four, 1-h didactic sessions on screening and logistically managing patients in need of inpatient or outpatient psychiatric care were created. Objectives were based on the complexities of state law, institutional policy, and expert opinion. All 15 residents in a family residency program participated in the didactic series. Resident confidence to complete all aspects of these scenarios was measured by questionnaire before (Time 1), immediately after (Time 2), and 5 months after completion of the didactic series (Time 3). Results: From Time 1 to Time 3, resident confidence significantly increased for 10/17 (58%) aspects of assessing and managing psychiatric crises (all P < .017). There was no evidence that confidence for any measure decreased from Time 2 to Time 3. Conclusions: The revised curriculum increased family medicine resident confidence to assess and manage outpatient psychiatric crises 5 months after the training was completed.

Background and Objectives: Preceptors in family medicine residencies need feedback to improve. When we found no validated, behavior-based tool to assess the outpatient precepting of family medicine residents, we sought to fill this gap by developing and initially validating the Mayo Outpatient Precepting Evaluation Tool (MOPET). Methods: To develop the MOPET, we applied the Stanford Faculty Development Program (SFDP) theoretical framework for education, more recent work on peer review of medical teaching, and expert review of items. The residency behavioral scientist and a volunteer physician independently completed the MOPET while co-observing a precepting physician during continuity clinic sessions (N=20). We assessed the tool’s validity via interrater reliability and cross-validation with the SFDP-26. Results: The tool demonstrated high interrater reliability for the following effective teaching behaviors: (a) allowing the resident to present without interrupting, (b) encouraging the formulation of a goal, (c) checking in on the resident’s goal, (d) using multimodal teaching aids, (e) asking to discuss the differential diagnosis, (f) asking to discuss alternative management, (g) encouraging the resident to pursue literature and/or other resources, and (h) reinforcing self-directed learning. The MOPET measures strongly correlated with most items from the SFDP-26, indicating good cross-validity. Conclusions: The MOPET is a theoretically sound, behavior-based, reliable, and initially validated tool for peer review of outpatient family medicine resident teaching. This tool can support faculty development in outpatient clinical learning environments.

Background and Objectives: Primary care physicians (PCPS) are increasingly responsible for managing mental health, which can involve assessment and management of a psychiatric crisis. Psychiatric crises can include acute suicidal or homicidal ideation and capacity-impairing psychosis. Evidence suggests PCPs do not consistently assess or manage psychiatric crises and it is unclear how to train PCPs to address these potentially lethal scenarios. The main objective was to increase PCP resident confidence in assessing and managing a range of psychiatric crises. Methods: In a family medicine residency program that trains PCPs, we developed a three, 1-h didactic series and point-of-care reference documents. The curriculum focused on screening, outpatient management, inpatient criteria, logistics of voluntary and involuntary admission, and legal considerations. Resident confidence was measured by questionnaire before and 3 months after curriculum completion. Results: Prior to training, residents did not feel confident in assessing and managing psychiatric crises, except a slight majority (62%) in screening for suicidal and homicidal ideation. Resident confidence significantly increased for every aspect of assessing and managing psychiatric crises after the training (all P-values < .05), with the largest improvements for further assessing hallucinations, delusions, and suicidal and homicidal ideation. Conclusions: As PCPs increasingly manage mental illness, they will encounter a range of psychiatric crises in clinic. This study demonstrates that a brief training intervention and point-of-care resources can significantly increase PCP confidence to assess and manage these urgent, dangerous scenarios.