Cook, David A.

Objectives Debriefing is a common feature of technology‐enhanced simulation ( TES ) education. However, evidence for its effectiveness remains unclear. We sought to characterise how debriefing is reported in the TES literature, identify debriefing features that are associated with improved outcomes, and evaluate the effectiveness of debriefing when combined with TES . Methods We systematically searched databases, including MEDLINE , EMBASE and Scopus, and reviewed previous bibliographies for original comparative studies investigating the use of TES with debriefing in training health care providers. Reviewers, in duplicate, evaluated study quality and abstracted information on instructional design, debriefing and outcomes. Effect sizes ( ES ) were pooled using random‐effects meta‐analysis. Results From 10 903 potentially eligible studies, we identified 177 studies (11 511 learners) that employed debriefing as part of TES . Key characteristics of debriefing (e.g. duration, educator presence and characteristics, content, structure/method, timing, use of video) were usually incompletely reported. A meta‐analysis of four studies demonstrated that video‐assisted debriefing has negligible and non‐significant effects for time skills ( ES = 0.10) compared with non‐video‐assisted debriefing. Meta‐analysis demonstrated non‐significant effects in favour of expert modelling with short debriefing in comparison with long debriefing (ES range = 0.21–0.74). Among studies comparing terminal with concurrent debriefing, results were variable depending on outcome measures and the context of training (e.g. medical resuscitation versus technical skills). Eight additional studies revealed insight into the roles of other debriefing‐related factors (e.g. multimedia debriefing, learner‐led debriefing, debriefing duration, content of debriefing). Among studies that compared simulation plus debriefing with no intervention, pooled ES s were favourable for all outcomes (ES range = 0.28–2.16). Conclusions Limited evidence suggests that video‐assisted debriefing yields outcomes similar to those of non‐video‐assisted debriefing. Other debriefing design features show mixed or non‐significant results. As debriefing characteristics are usually incompletely reported, future debriefing research should describe all the key debriefing characteristics along with their associated descriptors.

BACKGROUND AND OBJECTIVE: Pediatrics has embraced technology-enhanced simulation (TES) as an educational modality, but its effectiveness for pediatric education remains unclear. The objective of this study was to describe the characteristics and evaluate the effectiveness of TES for pediatric education. METHODS: This review adhered to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) standards. A systematic search of Medline, Embase, CINAHL, ERIC, Web of Science, Scopus, key journals, and previous review bibliographies through May 2011 and an updated Medline search through October 2013 were conducted. Original research articles in any language evaluating the use of TES for educating health care providers at any stage, where the content solely focuses on patients 18 years or younger, were selected. Reviewers working in duplicate abstracted information on learners, clinical topic, instructional design, study quality, and outcomes. We coded skills (simulated setting) separately for time and nontime measures and similarly classified patient care behaviors and patient effects. RESULTS: We identified 57 studies (3666 learners) using TES to teach pediatrics. Effect sizes (ESs) were pooled by using a random-effects model. Among studies comparing TES with no intervention, pooled ESs were large for outcomes of knowledge, nontime skills (eg, performance in simulated setting), behaviors with patients, and time to task completion (ES = 0.80–1.91). Studies comparing the use of high versus low physical realism simulators showed small to moderate effects favoring high physical realism (ES = 0.31–0.70). CONCLUSIONS: TES for pediatric education is associated with large ESs in comparison with no intervention. Future research should include comparative studies that identify optimal instructional methods and incorporate pediatric-specific issues into educational interventions.

Abstract Objectives Technology‐enhanced simulation is used frequently in emergency medicine ( EM ) training programs. Evidence for its effectiveness, however, remains unclear. The objective of this study was to evaluate the effectiveness of technology‐enhanced simulation for training in EM and identify instructional design features associated with improved outcomes by conducting a systematic review. Methods The authors systematically searched MEDLINE , EMBASE , CINAHL , ERIC , Psych INFO , Scopus, key journals, and previous review bibliographies through May 2011. Original research articles in any language were selected if they compared simulation to no intervention or another educational activity for the purposes of training EM health professionals (including student and practicing physicians, midlevel providers, nurses, and prehospital providers). Reviewers evaluated study quality and abstracted information on learners, instructional design (curricular integration, feedback, repetitive practice, mastery learning), and outcomes. Results From a collection of 10,903 articles, 85 eligible studies enrolling 6,099 EM learners were identified. Of these, 56 studies compared simulation to no intervention, 12 compared simulation with another form of instruction, and 19 compared two forms of simulation. Effect sizes were pooled using a random‐effects model. Heterogeneity among these studies was large (I 2 ≥ 50%). Among studies comparing simulation to no intervention, pooled effect sizes were large (range = 1.13 to 1.48) for knowledge, time, and skills and small to moderate for behaviors with patients (0.62) and patient effects (0.43; all p < 0.02 except patient effects p = 0.12). Among comparisons between simulation and other forms of instruction, the pooled effect sizes were small (≤0.33) for knowledge, time, and process skills (all p > 0.1). Qualitative comparisons of different simulation curricula are limited, although feedback, mastery learning, and higher fidelity were associated with improved learning outcomes. Conclusions Technology‐enhanced simulation for EM learners is associated with moderate or large favorable effects in comparison with no intervention and generally small and nonsignificant benefits in comparison with other instruction. Future research should investigate the features that lead to effective simulation‐based instructional design.

AbstractA plenary panel session at the 2012 Academic Emergency Medicine consensus conference “Education Research in Emergency Medicine: Opportunities, Challenges, and Strategies for Success” discussed barriers educators face in imagining, designing, and implementing studies to address educational challenges. This proceedings article presents a general approach to getting started in education research. Four examples of studies from the medical education literature that illustrate a distinct way to approach specific research questions are discussed. The study designs used are applicable to a variety of education research problems in emergency medicine (EM). Potential applications of studies are discussed, as well as effects and lessons learned.