Lawson, Luan

Purpose To examine validity evidence for a standardized patient scenario assessing medical students’ ability to promote value using patient-centered communication (in response to a patient requesting an unnecessary test) and to explore the potential effect of various implementation and curricular factors on student scores. Method Third-year medical students (N = 516) from 5 U.S. MD-granting medical schools completed the communication scenario between 2014 and 2017 as part of a larger objective structured clinical examination (OSCE). Centralized raters assessed performance using an 11-item checklist. The authors collected multiple sources of validity evidence. Results The mean checklist score was 0.85 (standard deviation 0.09). Interrater reliability for checklist scores was excellent (0.87, 95% confidence interval = 0.78–0.93). Generalizability and Phi-coefficients were, respectively, 0.65 and 0.57. Scores decreased as the number of OSCE stations increased ( r = −0.15, P = .001) and increased when they were used for summative purposes ( r = 0.26, P < .001). Scores were not associated with curricular time devoted to high-value care ( r = 0.02, P = .67) and decreased when more clerkships were completed before the assessment ( r = −0.12, P = .006). Conclusions This multisite study provides validity evidence supporting the use of scenario scores to assess the ability of medical students to promote value in clinical encounters using patient-centered communication. Findings illuminate the potential effect of OSCE structure and purpose on student performance and suggest clerkship learning experiences may not reinforce what students are taught in the formal curriculum regarding high-value care. Devoting more time to the topic appears insufficient to counteract this erosion.

Educators, policy makers, and health systems leaders are calling for significant reform of undergraduate medical education (UME) and graduate medical education (GME) programs to meet the evolving needs of the health care system. Nationally, several schools have initiated innovative curricula in both classroom and workplace learning experiences to promote education in health systems science (HSS), which includes topics such as value-based care, health system improvement, and population and public health. However, the successful implementation of HSS curricula across schools is challenged by issues of curriculum design, assessment, culture, and accreditation, among others. In this report of a working conference using thematic analysis of workshop recommendations and experiences from 11 U.S. medical schools, the authors describe seven priority areas for the successful integration and sustainment of HSS in educational programs, and associated challenges and potential solutions. In 2015, following regular HSS workgroup phone calls and an Accelerating Change in Medical Education consortium-wide meeting, the authors identified the priority areas: partner with licensing, certifying, and accrediting bodies; develop comprehensive, standardized, and integrated curricula; develop, standardize, and align assessments; improve the UME to GME transition; enhance teachers’ knowledge and skills, and incentives for teachers; demonstrate value added to the health system; and address the hidden curriculum. These priority areas and their potential solutions can be used by individual schools and HSS education collaboratives to further outline and delineate the steps needed to create, deliver, study, and sustain effective HSS curricula with an eye toward integration with the basic and clinical sciences curricula.

Purpose The authors performed a review of 30 Accelerating Change in Medical Education full grant submissions and an analysis of the health systems science (HSS)-related curricula at the 11 grant recipient schools to develop a potential comprehensive HSS curricular framework with domains and subcategories. Method In phase 1, to identify domains, grant submissions were analyzed and coded using constant comparative analysis. In phase 2, a detailed review of all existing and planned syllabi and curriculum documents at the grantee schools was performed, and content in the core curricular domains was coded into subcategories. The lead investigators reviewed and discussed drafts of the categorization scheme, collapsed and combined domains and subcategories, and resolved disagreements via group discussion. Results Analysis yielded three types of domains: core, cross-cutting, and linking. Core domains included health care structures and processes; health care policy, economics, and management; clinical informatics and health information technology; population and public health; value-based care; and health system improvement. Cross-cutting domains included leadership and change agency; teamwork and interprofessional education; evidence-based medicine and practice; professionalism and ethics; and scholarship. One linking domain was identified: systems thinking. Conclusions This broad framework aims to build on the traditional definition of systems-based practice and highlight the need for medical and other health professions schools to better align education programs with the anticipated needs of the systems in which students will practice. HSS will require a critical investigation into existing curricula to determine the most efficient methods for integration with the basic and clinical sciences.