Price, David W.

Background and Objectives: Evidence on the relationship between formative assessment engagement and summative assessment outcomes in practicing physicians is sparse. We evaluated the relationship between engagement in the American Board of Family Medicine (ABFM) formative Continuous Knowledge Self-Assessment (CKSA) and performance on high-stakes summative assessments. Methods: This retrospective cohort study included 24,926 ABFM diplomates who completed CKSA modules and summative assessments between 2017 and 2023. We analyzed CKSA engagement metrics—such as the number of quarters completed, time of completion, and self-reported confidence—against performance on summative assessments, measured by z scores. Multivariable regression models controlled for demographic factors and prior assessment performance. Results: The overall cohort summative assessment pass rate during the study period was 90.3%. Greater CKSA engagement was strongly associated with higher summative assessment performance. Diplomates who completed all four CKSA quarters had significantly higher summative assessment z scores than those completing fewer quarters (P<.001). Early CKSA completion and spending more time on low-confidence questions were also positively correlated with both CKSA and summative assessment scores (P<.001). These effects were observed across different levels of prior exam performance. Conclusions: Engagement in formative assessments like CKSA, particularly early and consistent participation and reviewing incorrect or low-confidence questions, is linked to better outcomes on high-stakes assessments. Future research should explore the mechanisms underlying these associations and consider developing an index of engagement to identify physicians at risk of poor performance. Incorporating structured, longitudinal self-assessments like CKSA into certification requirements could enhance continuous learning and improve summative exam readiness.

Purpose To determine the past experiences with, current use of, and anticipated use of online learning and simulation-based education among practicing U.S. physicians, and how findings vary by age. Method The authors surveyed 4,648 randomly sampled board-certified U.S. physicians, September 2015 to April 2016, using Internet-based and paper questionnaires. Survey items (some optional) addressed past and current technology usage, perceived technology effectiveness, and anticipated future use of specific technology innovations. Results Of 988 respondents, 444 completed optional items. Of these, 429/442 (97.1%) had used online learning and 372/442 (84.2%) had used simulation-based education in the past five years. Desire for more online learning was modest (mean [standard deviation], 4.6 [1.5]; 1 = strongly disagree, 7 = strongly agree), as was desire for more simulation-based education (4.2 [1.7]). Both online learning and simulation-based education were perceived as effective (5.2 [1.4]; 5.0 [1.4]). Physicians believed they possess adequate skills for online learning (5.8 [1.2]) and that point-of-care learning is vital to effective patient care (5.3 [1.3]). Only 39.0% used objective performance data to guide their learning choices, although 64.6% agreed that such information would be useful. The highest-rated innovations included a central repository for listing educational opportunities and tracking continuing education credits, an app to award credit for answering patient-focused questions, 5-minute and 20-minute clinical updates, and an e-mailed “question of the week.” Responses to most survey items were similar across age groups. Conclusions Practicing physicians generally seem receptive and prepared to use a variety of educational technologies, regardless of age.

Purpose Professional development (PD)—both for-credit continuing medical education (CME) and informal self-directed or point-of-care learning—is vital to all physicians. The authors sought to understand physicians’ PD perceptions and practices and how these vary by specialty and practice type. Method The authors administered an Internet and paper survey, from September 2015 to April 2016, to randomly sampled U.S. physicians. Survey items addressed perceived PD needs and barriers and how physicians identify knowledge/skills gaps. Results Of 4,648 invitees, 988 (21.6%) responded. Respondents believed that they already know what they need to learn (mean 5.8 [1 = strongly disagree; 7 = strongly agree]), can answer clinical questions using available resources (5.9), and want credit for learning during patient care (5.1). They did not strongly desire help identifying learning gaps (4.0) or indicate difficulty accumulating CME credits (3.1). Most PD was done during personal time (5.5). Competencies regarding medical knowledge/skills, wellness, informatics, and practice/systems improvement were rated the highest priority, while research, teaching, and professionalism were rated the lowest. The most important sources used to identify knowledge/skills gaps were immediate patient care needs (4.1 [1 = not important; 5 = extremely important]), personal awareness (3.8), and practice updates (3.7). The most important barriers were time (3.5) and cost (2.9). Differences by specialty and practice type were generally small and not statistically significant. Conclusions Physicians feel confident in identifying their own learning needs, perceive medical knowledge/skills as their highest-priority need, and desire more credit for learning during patient care.