Isaak, Robert S.

BACKGROUND: Burnout remains prevalent among anesthesiology residents. Contributing workplace factors include limited control over scheduling, unpredictable hours, critical patient care needs, and other residency challenges. The Family Anesthesia Experience (FAX) is a program designed to educate residents and their support persons on wellness and residency demands. The focus of this multisite study was to evaluate the impact of FAX on first year clinical anesthesiology (CA1, ie, PGY2s) residents’ wellness. METHODS: This was a prospective evaluation study involving 15 US anesthesiology residency programs, 13 of which completed FAX. CA1 wellness was evaluated through surveys assessing perceived stress as the primary outcome, with exploratory outcomes including burnout, depression, psychological wellness, and social support. Additionally, changes in support persons’ understanding and knowledge of residency demands were tracked over time. The study included a historical control group of 302 CA1s (2019-2021) who did not participate in FAX, and an intervention group of 88 CA1s that participated in FAX (2021-2022). Statistical analyses compared changes in metrics between groups using linear mixed models and generalized linear models. RESULTS: Significant reductions in perceived stress were observed in the intervention group at 6 months post-FAX (mean change = –2.14, 95% CI, –3.95 to –0.34; P = .021, effect size = 0.58) relative to the control group, but there was no sustained difference at 12 months. Compared to the control support persons, intervention support persons reported increased understanding of residency demands, with significant improvements at 6 (mean change = 2.80, 95% CI, 1.12–4.48; P = .001, effect size = 0.77) and 12 (mean change = 2.59, 95% CI, 0.12–5.05; P = .039, effect size = 0.71) months. Improvement in support persons’ understanding did not predict changes in CA1 wellness outcomes. First generation physician residents reported lower family support at baseline but showed significant improvements in family support at 6 months (mean change = 3.05 points; 95% CI, 0.25–5.84; P = .033; effect size = 1.23) compared to their non first generation peers. CONCLUSIONS: FAX participation was associated with reduced perceived stress in residents up to 6 months post-intervention. First generation physicians experienced greater improvements in perceived family support compared to their non first generation peers. Our findings suggest that resident wellness may be enhanced by engaging both residents and their support persons in an educational wellness program that outlines the professional expectations and demands of anesthesiology residency training.

ABSTRACT Introduction Lack of social support negatively impacts medical trainees' wellness. Programmes to educate medical trainees and their support persons (SPs) have been developed to improve resident wellness, but implementation of these programmes at other institutions remains unclear. We aimed to demonstrate the feasibility of implementing the Family Anesthesia Experience (FAX) programme across multiple institutions and to assess the programme's utility. Methods FAX includes didactics on wellness, burnout, substance use disorder and institutional resources; a panel of senior residents and their SPs; and small‐group simulations of airway management, common procedures and a high‐fidelity intraoperative code. We assessed the programme qualitatively using semistructured interviews of 20 first year anaesthesiology residents and 20 SPs. Data analysis was performed via a content analysis approach. Results Twelve anaesthesiology programmes of varying size and geographic locations implemented FAX with minor adaptations. Minimal direct costs were incurred for the virtual programme. Participants enjoyed the event overall. Content analysis showed that FAX improved SPs' understanding of residency and communication between SPs and residents, had minimal impact on SPs' stress, reduced residents' interpersonal stress with SPs and modified perceptions on wellness and burnout. We developed a conceptual model in which we propose SPs' understanding and empathy for their residents would improve through FAX, which may improve communication with and support for residents. Conclusion We demonstrated that FAX can be implemented at varying anaesthesiology residency programmes. Family‐oriented educational events like FAX may be an effective approach to improve SPs' understanding of a resident's role, thereby increasing social support for residents.

BACKGROUND: In 2018, a set of entrustable professional activities (EPAs) and procedural skills assessments were developed for anesthesiology training, but they did not assess all the Accreditation Council for Graduate Medical Education (ACGME) milestones. The aims of this study were to (1) remap the 2018 EPA and procedural skills assessments to the revised ACGME Anesthesiology Milestones 2.0, (2) develop new assessments that combined with the original assessments to create a system of assessment that addresses all level 1 to 4 milestones, and (3) provide evidence for the validity of the assessments. METHODS: Using a modified Delphi process, a panel of anesthesiology education experts remapped the original assessments developed in 2018 to the Anesthesiology Milestones 2.0 and developed new assessments to create a system that assessed all level 1 through 4 milestones. Following a 24-month pilot at 7 institutions, the number of EPA and procedural skill assessments and mean scores were computed at the end of the academic year. Milestone achievement and subcompetency data for assessments from a single institution were compared to scores assigned by the institution’s clinical competency committee (CCC). RESULTS: New assessment development, 2 months of testing and feedback, and revisions resulted in 5 new EPAs, 11 nontechnical skills assessments (NTSAs), and 6 objective structured clinical examinations (OSCEs). Combined with the original 20 EPAs and procedural skills assessments, the new system of assessment addresses 99% of level 1 to 4 Anesthesiology Milestones 2.0. During the 24-month pilot, aggregate mean EPA and procedural skill scores significantly increased with year in training. System subcompetency scores correlated significantly with 15 of 23 (65.2%) corresponding CCC scores at a single institution, but 8 correlations (36.4%) were <30.0, illustrating poor correlation. CONCLUSIONS: A panel of experts developed a set of EPAs, procedural skill assessment, NTSAs, and OSCEs to form a programmatic system of assessment for anesthesiology residency training in the United States. The method used to develop and pilot test the assessments, the progression of assessment scores with time in training, and the correlation of assessment scores with CCC scoring of milestone achievement provide evidence for the validity of the assessments.

BACKGROUND: The American Board of Anesthesiology administers the APPLIED Examination as a part of initial certification, which as of 2018 includes 2 components—the Standardized Oral Examination (SOE) and the Objective Structured Clinical Examination (OSCE). The goal of this study is to investigate the measurement construct(s) of the APPLIED Examination to assess whether the SOE and the OSCE measure distinct constructs (ie, factors). METHODS: Exploratory item factor analysis of candidates’ performance ratings was used to determine the number of constructs, and confirmatory item factor analysis to estimate factor loadings within each construct and correlation(s) between the constructs. RESULTS: In exploratory item factor analysis, the log-likelihood ratio test and Akaike information criterion index favored the 3-factor model, with factors reflecting the SOE, OSCE Communication and Professionalism, and OSCE Technical Skills. The Bayesian information criterion index favored the 2-factor model, with factors reflecting the SOE and the OSCE. In confirmatory item factor analysis, both models suggest moderate correlation between the SOE factor and the OSCE factor; the correlation was 0.49 (95% confidence interval [CI], 0.42-0.55) for the 3-factor model and 0.61 (95% CI, 0.54-0.64) for the 2-factor model. The factor loadings were lower for Technical Skills stations of the OSCE (ranging from 0.11 to 0.25) compared with those of the SOE and Communication and Professionalism stations of the OSCE (ranging from 0.36 to 0.50). CONCLUSIONS: The analyses provide evidence that the SOE and the OSCE measure distinct constructs, supporting the rationale for administering both components of the APPLIED Examination for initial certification in anesthesiology.

In 2018, the American Board of Anesthesiology (ABA) became the first US medical specialty certifying board to incorporate an Objective Structured Clinical Examination (OSCE) into its initial certification examination system. Previously, the ABA’s staged examination system consisted of 2 written examinations (the BASIC and ADVANCED examinations) and the Standardized Oral Examination (SOE). The OSCE and the existing SOE are now 2 separate components of the APPLIED Examination. This report presents the results of the first-year OSCE administration. A total of 1410 candidates took both the OSCE and the SOE in 2018. Candidate performance approximated a normal distribution for both the OSCE and the SOE, and was not associated with the timing of the examination, including day of the week, morning versus afternoon session, and order of the OSCE and the SOE. Practice-based Learning and Improvement was the most difficult station, while Application of Ultrasonography was the least difficult. The correlation coefficient between SOE and OSCE scores was 0.35 ([95% confidence interval {CI}, 0.30–0.39]; P < .001). Scores for the written ADVANCED Examination were modestly correlated with scores for the SOE ( r = 0.29 [95% CI, 0.25–0.34]; P < .001) and the OSCE ( r = 0.15 [95% CI, 0.10–0.20]; P < .001). Most of the candidates who failed the SOE passed the OSCE, and most of the candidates who failed the OSCE passed the SOE. Of the 1410 candidates, 77 (5.5%) failed the OSCE, 155 (11.0%) failed the SOE, and 25 (1.8%) failed both. Thus, 207 (14.7%) failed at least 1 component of the APPLIED Examination. Adding an OSCE to a board certification examination system is feasible. Preliminary evidence indicates that the OSCE measures aspects of candidate abilities distinct from those measured by other examinations used for initial board certification.

With its first administration of an Objective Structured Clinical Examination (OSCE) in 2018, the American Board of Anesthesiology (ABA) became the first US medical specialty certifying board to incorporate this type of assessment into its high-stakes certification examination system. The fundamental rationale for the ABA’s introduction of the OSCE is to include an assessment that allows candidates for board certification to demonstrate what they actually “do” in domains relevant to clinical practice. Inherent in this rationale is that the OSCE will capture competencies not well assessed in the current written and oral examinations—competencies that will allow the ABA to judge whether a candidate meets the standards expected for board certification more properly. This special article describes the ABA’s journey from initial conceptualization through first administration of the OSCE, including the format of the OSCE, the process for scenario development, the standardized patient program that supports OSCE administration, examiner training, scoring, and future assessment of reliability, validity, and impact of the OSCE. This information will be beneficial to both those involved in the initial certification process, such as residency graduate candidates and program directors, and others contemplating the use of high-stakes summative OSCE assessments.