Hamstra, Stanley J

Purpose To identify common and overlapping themes among the interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), professionalism (PROF), and systems-based practice (SBP) milestones of the transitional year and 26 specialties. Method In May 2017, milestones were accessed from the Accreditation Council for Graduate Medical Education specialties website. A thematic analysis of the ICS, PBLI, PROF, and SBP milestones was performed to determine unique and common themes across these competencies and across specialties. Keywords from the common program requirements were initially applied as codes to the milestones. Codes were then grouped into common themes. Results Twenty-two themes were identified: 15 (68%) were unique to a given competency (3 related to ICS, 4 related to PBLI, 5 related to PROF, and 3 related to SBP), and 7 (32%) appeared in the milestones of more than one core competency. Eleven themes (50%) were used by 20 or more specialties, and 6 themes (27%) by 10 or fewer specialties. No theme was present across all specialties. Conclusions The ICS, PBLI, PROF, and SBP milestones contain multiple themes with areas of overlap among these four competencies and substantial variability across specialties. This variability may create differential expectations of residents across specialties, complicate faculty development, and make sharing assessment tools difficult. The thematic analysis provides important insights into how individual specialties interpret and operationalize the ICS, PBLI, PROF, and SBP competency domains and can inform future revisions of milestones to enable harmonization and shared understanding of these competencies across specialties where appropriate.

Context Complete reporting of research is essential to enable consumers to accurately appraise, interpret and apply findings. Quality appraisal checklists are giving way to tools that judge the risk for bias. Objectives We sought to determine the prevalence of these complementary aspects of research reports (completeness of reporting and perceived risk for bias) of randomised studies in health professions education. Methods We searched bibliographic databases for randomised studies of health professions education. We appraised two cohorts representing different time periods (2008–2010 and 2014, respectively) and worked in duplicate to apply the CONSORT guidelines and Cochrane Risk of Bias tool. We explored differences between time periods using independent‐samples t ‐tests or the chi‐squared test, as appropriate. Results We systematically identified 180 randomised studies (2008–2010, n = 150; 2014, n = 30). Frequencies of reporting of CONSORT elements within full‐text reports were highly variable and most elements were reported in fewer than 50% of studies. We found a statistically significant difference in the CONSORT reporting index (maximum score: 500) between the 2008–2010 (mean ± standard deviation [SD]: 242.7 ± 55.6) and 2014 (mean ± SD: 311.6 ± 53.2) cohorts (p < 0.001). High or unclear risk for bias was most common for allocation concealment (157, 87%) and blinding of participants (147, 82%), personnel (152, 84%) and outcome assessors (112, 62%). Most risk for bias elements were judged to be unclear (range: 51–84%). Risk for bias elements significantly improved over time for blinding of participants (p = 0.007), incomplete data (p < 0.001) and the presence of other sources of bias (p < 0.001). Conclusions Reports of randomised studies in health professions education frequently omit elements recommended by the CONSORT statement. Most reports were assessed as having a high or unclear risk for bias. Greater attention to how studies are reported at study outset and in manuscript preparation could improve levels of complete transparent reporting.

Context Self‐regulated learning ( SRL ) requires an active learner who has developed a set of processes for managing the achievement of learning goals. Simulation‐based training is one context in which trainees can safely practise learning how to learn. Objectives The purpose of the present study was to evaluate, in the simulation‐based training context, the effectiveness of interventions designed to support trainees in SRL activities. We used the social‐cognitive model of SRL to guide a systematic review and meta‐analysis exploring the links between instructor supervision, supports or scaffolds for SRL , and educational outcomes. Methods We searched databases including MEDLINE and Scopus, and previous reviews, for material published until December 2011. Studies comparing simulation‐based SRL interventions with another intervention for teaching health professionals were included. Reviewers worked independently and in duplicate to extract information on learners, study quality and educational outcomes. We used random‐effects meta‐analysis to compare the effects of supervision (instructor present or absent) and SRL educational supports (e.g. goal‐setting study guides present or absent). Results From 11 064 articles, we included 32 studies enrolling 2482 trainees. Only eight of the 32 studies included educational supports for SRL . Compared with instructor‐supervised interventions, unsupervised interventions were associated with poorer immediate post‐test outcomes (pooled effect size: −0.34, p = 0.09; n = 19 studies) and negligible effects on delayed (i.e. > 1 week) retention tests (pooled effect size: 0.11, p = 0.63; n = 8 studies). Interventions including SRL supports were associated with small benefits compared with interventions without supports on both immediate post‐tests (pooled effect size: 0.23, p = 0.22; n = 5 studies) and delayed retention tests (pooled effect size: 0.44, p = 0.067; n = 3 studies). Conclusions Few studies in the simulation literature have designed SRL training to explicitly support trainees’ capacity to self‐regulate their learning. We recommend that educators and researchers shift from thinking about SRL as learning alone to thinking of SRL as comprising a shared responsibility between the trainee and the instructional designer (i.e. learning using designed supports that help prepare individuals for future learning).

AbstractA plenary panel session at the 2012 Academic Emergency Medicine consensus conference “Education Research in Emergency Medicine: Opportunities, Challenges, and Strategies for Success” discussed barriers educators face in imagining, designing, and implementing studies to address educational challenges. This proceedings article presents a general approach to getting started in education research. Four examples of studies from the medical education literature that illustrate a distinct way to approach specific research questions are discussed. The study designs used are applicable to a variety of education research problems in emergency medicine (EM). Potential applications of studies are discussed, as well as effects and lessons learned.