Macario, Alex

Background: Longitudinal assessments are increasingly common as a component of continuing certification programs offered by medical specialty boards. The goal of this study was to understand anesthesiologists’ utilization pattern, performance, and perception of the longitudinal assessment program administered by the American Board of Anesthesiology (Raleigh, North Carolina). Methods: The authors analyzed assessment usage and survey data from all anesthesiologists who participated in the Maintenance of Certification in Anesthesiology (MOCA) Minute from January 2018 to December 2019. Results: During the 24-month study period, 25,404 anesthesiologists completed all 120 questions assigned in at least 1 calendar year, collectively 5,801,160 responses to 1,646 unique questions. Participants were more likely to complete questions at the end of each quarter, with the highest frequency at the end of the year. Seventy-five percent of responses were first-time attempts, and 17% were first repeats. The median interval between original question presentation and first repeat was 314 days. The mean ± SD percent correct score was 68.3 ± 8.0% for first-time questions and 75.5 ± 12.6% for first repeats. The mean ± SD time to answer a first-time question was 26.8 ± 13.1 s, with shorter times for questions with correct than incorrect responses (25.0 ± 12.7 vs . 30.7 ± 13.3 s; time difference, −5.69 s; 95% CI, −5.71 to −5.66; P < 0.001). Of 5.11 million responses with ratings of relevance (88.2% of all responses analyzed), 4.86 million (95.2%) were felt to be somewhat or very relevant to practice. The percent correct scores were higher for responses with higher relevance and confidence ratings. For responses rated “not relevant,” “somewhat relevant,” and “very relevant,” the percent correct was 58.5%, 67.6%, and 77.0%, respectively; for responses rated as “unsure,” “somewhat confident,” and “very confident,” the percent correct was 53.6%, 67.9%, and 84.4%, respectively. Conclusions: MOCA Minute was generally used as designed. Evidence of learning was suggested by improved performance on repeat questions.

BACKGROUND: The American Board of Anesthesiology administers the APPLIED Examination as a part of initial certification, which as of 2018 includes 2 components—the Standardized Oral Examination (SOE) and the Objective Structured Clinical Examination (OSCE). The goal of this study is to investigate the measurement construct(s) of the APPLIED Examination to assess whether the SOE and the OSCE measure distinct constructs (ie, factors). METHODS: Exploratory item factor analysis of candidates’ performance ratings was used to determine the number of constructs, and confirmatory item factor analysis to estimate factor loadings within each construct and correlation(s) between the constructs. RESULTS: In exploratory item factor analysis, the log-likelihood ratio test and Akaike information criterion index favored the 3-factor model, with factors reflecting the SOE, OSCE Communication and Professionalism, and OSCE Technical Skills. The Bayesian information criterion index favored the 2-factor model, with factors reflecting the SOE and the OSCE. In confirmatory item factor analysis, both models suggest moderate correlation between the SOE factor and the OSCE factor; the correlation was 0.49 (95% confidence interval [CI], 0.42-0.55) for the 3-factor model and 0.61 (95% CI, 0.54-0.64) for the 2-factor model. The factor loadings were lower for Technical Skills stations of the OSCE (ranging from 0.11 to 0.25) compared with those of the SOE and Communication and Professionalism stations of the OSCE (ranging from 0.36 to 0.50). CONCLUSIONS: The analyses provide evidence that the SOE and the OSCE measure distinct constructs, supporting the rationale for administering both components of the APPLIED Examination for initial certification in anesthesiology.

BACKGROUND: Anesthesiology residents’ experiences and perspectives about their programs may be helpful in improving training. The goals of this repeated cross-sectional survey study are to determine: (1) the most important factors residents consider in choosing an anesthesiology residency, (2) the aspects of the clinical base year that best prepare residents for anesthesia clinical training, and what could be improved, (3) whether residents are satisfied with their anesthesiology residency and what their primary struggles are, and (4) whether residents believe their residency prepares them for proficiency in the 6 Accreditation Council for Graduate Medical Education (ACGME) Core Competencies and for independent practice. METHODS: Anesthesiologists beginning their US residency training from 2013 to 2016 were invited to participate in anonymous, confidential, and voluntary self-administered online surveys. Resident cohort was defined by clinical anesthesia year 1, such that 9 survey administrations were included in this study—3 surveys for the 2013 and 2014 cohorts (clinical anesthesia years 1–3), 2 surveys for the 2015 cohort (clinical anesthesia years 1–2), and 1 survey for the 2016 cohort (clinical anesthesia year 1). RESULTS: The overall response rate was 36% (4707 responses to 12,929 invitations). On a 5-point Likert scale with 1 as “very unimportant” and 5 as “very important,” quality of clinical experience (4.7–4.8 among the cohorts) and departmental commitment to education (4.3–4.5) were rated as the most important factors in anesthesiologists’ choice of residency. Approximately 70% of first- and second-year residents agreed that their clinical base year prepared them well for anesthesiology residency, particularly clinical training experiences in critical care rotations, anesthesiology rotations, and surgery rotations/perioperative procedure management. Overall, residents were satisfied with their choice of anesthesiology specialty (4.4–4.5 on a 5-point scale among cohort-training levels) and their residency programs (4.0–4.1). The residency training experiences mostly met their expectations (3.8–4.0). Senior residents who reported any struggles highlighted academic more than interpersonal or technical difficulties. Senior residents generally agreed that the residency adequately prepared them for independent practice (4.1–4.4). Of the 6 ACGME Core Competencies, residents had the highest confidence in professionalism (4.7–4.9) and interpersonal and communication skills (4.6–4.8). Areas in residency that could be improved include the provision of an appropriate balance between education and service and allowance for sufficient time off to search and interview for a postresidency position. CONCLUSIONS: Anesthesiology residents in the United States indicated they most value quality of clinical training experiences and are generally satisfied with their choice of specialty and residency program.

BACKGROUND: A temporary decrease in anesthesiology residency graduates that occurred around the turn of the millennium may have workforce implications. The aims of this study are to describe, between 2005 and 2015, (1) demographic changes in the workforce of physicians trained as anesthesiologists; (2) national and state densities of these physicians, as well as temporal changes in the densities; and (3) retention of medical licenses by mid- and later-career anesthesiologists. METHODS: Using records from the American Board of Anesthesiology and state medical and osteopathic boards, the numbers of licensed physicians aged 30–59 years who had completed Accreditation Council for Graduate Medical Education–accredited anesthesiology residency training were calculated cross-sectionally for 2005, 2010, and 2015. Demographic trends were then described. Census data were used to calculate national and state densities of licensed physicians. Individual longitudinal data were used to describe retention of medical licenses among older physicians. RESULTS: The number of licensed physicians trained as anesthesiologists aged 30–59 years increased from 32,644 in 2005 to 36,543 in 2010 and 36,624 in 2015, representing a national density of 1.10, 1.18, and 1.14 per 10,000 population in those years, respectively. The density of anesthesiologists among states ranged from 0.37 to 3.10 per 10,000 population. The age distribution differed across the years. For example, anesthesiologists aged 40–49 years predominated in 2005 (47%), but by 2015, only 31% of anesthesiologists were aged 40–49 years. The proportion of female anesthesiologists grew from 22% in 2005, to 24% in 2010, and to 28% in 2015, particularly among early-career anesthesiologists. For anesthesiologists with licenses in 2005, the number who still had active licenses in 2015 decreased by 9.6% for those aged 45–49 years, by 14.1% for those aged 50–54 years, and by 19.7% for those aged 55–59 years. CONCLUSIONS: The temporary decrease in anesthesiology residency graduates around the turn of the 21st century decreased the proportion of anesthesiologists who were midcareer as of 2015. This may affect the future availability of senior leaders as well as the future overall workforce in the specialty as older anesthesiologists retire. National efforts to plan for workforce needs should recognize the geographical variability in the distribution of anesthesiologists.

In 2018, the American Board of Anesthesiology (ABA) became the first US medical specialty certifying board to incorporate an Objective Structured Clinical Examination (OSCE) into its initial certification examination system. Previously, the ABA’s staged examination system consisted of 2 written examinations (the BASIC and ADVANCED examinations) and the Standardized Oral Examination (SOE). The OSCE and the existing SOE are now 2 separate components of the APPLIED Examination. This report presents the results of the first-year OSCE administration. A total of 1410 candidates took both the OSCE and the SOE in 2018. Candidate performance approximated a normal distribution for both the OSCE and the SOE, and was not associated with the timing of the examination, including day of the week, morning versus afternoon session, and order of the OSCE and the SOE. Practice-based Learning and Improvement was the most difficult station, while Application of Ultrasonography was the least difficult. The correlation coefficient between SOE and OSCE scores was 0.35 ([95% confidence interval {CI}, 0.30–0.39]; P < .001). Scores for the written ADVANCED Examination were modestly correlated with scores for the SOE ( r = 0.29 [95% CI, 0.25–0.34]; P < .001) and the OSCE ( r = 0.15 [95% CI, 0.10–0.20]; P < .001). Most of the candidates who failed the SOE passed the OSCE, and most of the candidates who failed the OSCE passed the SOE. Of the 1410 candidates, 77 (5.5%) failed the OSCE, 155 (11.0%) failed the SOE, and 25 (1.8%) failed both. Thus, 207 (14.7%) failed at least 1 component of the APPLIED Examination. Adding an OSCE to a board certification examination system is feasible. Preliminary evidence indicates that the OSCE measures aspects of candidate abilities distinct from those measured by other examinations used for initial board certification.

With its first administration of an Objective Structured Clinical Examination (OSCE) in 2018, the American Board of Anesthesiology (ABA) became the first US medical specialty certifying board to incorporate this type of assessment into its high-stakes certification examination system. The fundamental rationale for the ABA’s introduction of the OSCE is to include an assessment that allows candidates for board certification to demonstrate what they actually “do” in domains relevant to clinical practice. Inherent in this rationale is that the OSCE will capture competencies not well assessed in the current written and oral examinations—competencies that will allow the ABA to judge whether a candidate meets the standards expected for board certification more properly. This special article describes the ABA’s journey from initial conceptualization through first administration of the OSCE, including the format of the OSCE, the process for scenario development, the standardized patient program that supports OSCE administration, examiner training, scoring, and future assessment of reliability, validity, and impact of the OSCE. This information will be beneficial to both those involved in the initial certification process, such as residency graduate candidates and program directors, and others contemplating the use of high-stakes summative OSCE assessments.

BACKGROUND: In January 2016, as part of the Maintenance of Certification in Anesthesiology (MOCA) program, the American Board of Anesthesiology launched MOCA Minute, a web-based longitudinal assessment, to supplant the former cognitive examination. We investigated the association between participation and performance in MOCA Minute and disciplinary actions against medical licenses of anesthesiologists. METHODS: All anesthesiologists with time-limited certificates (ie, certified in 2000 or after) who were required to register for MOCA Minute in 2016 were followed up through December 31, 2016. The incidence of postcertification prejudicial license actions was compared between those who did and did not register and compared between registrants who did and did not meet the MOCA Minute performance standard. RESULTS: The cumulative incidence of license actions was 1.2% (245/20,006) in anesthesiologists required to register for MOCA Minute. Nonregistration was associated with a higher incidence of license actions (hazard ratio, 2.93 [95% confidence interval {CI}, 2.15–4.00]). For the 18,534 (92.6%) who registered, later registration (after June 30, 2016) was associated with a higher incidence of license actions. In 2016, 16,308 (88.0%) anesthesiologists met the MOCA Minute performance standard. Of those not meeting the standard (n = 2226), most (n = 2093, 94.0%) failed because they did not complete the required 120 questions. Not meeting the standard was associated with a higher incidence of license actions (hazard ratio, 1.92 [95% CI, 1.36–2.72]). CONCLUSIONS: Both timely participation and meeting performance standard in MOCA Minute are associated with a lower likelihood of being disciplined by a state medical board.