Sorenson, Eric J.

AbstractIntroduction/AimsGraduate medical education programs must ensure residents and fellows acquire skills needed for independent practice. Workplace‐based observational assessments are informative but can be time‐ and resource‐intensive. In this study we sought to gather “relations‐to‐other‐variables” validity evidence for scores generated by the Electromyography Direct Observation Tool (EMG‐DOT) to inform its use as a measure of electrodiagnostic skill acquisition.MethodsScores on multiple assessments were compiled by trainees during Clinical Neurophysiology and Electromyography rotations at a large US academic medical center. Relationships between workplace‐based EMG‐DOT scores (n = 298) and scores on a prerequisite simulated patient exercise, patient experience surveys (n = 199), end‐of‐rotation evaluations (n = 301), and an American Association of Neuromuscular & Electrodiagnostic Medicine (AANEM) self‐assessment examination were assessed using Pearson correlations.ResultsAmong 23 trainees, EMG‐DOT scores assigned by physician raters correlated positively with end‐of‐rotation evaluations (r = 0.63, P = .001), but EMG‐DOT scores assigned by technician raters did not (r = 0.10, P = .663). When physician and technician ratings were combined, higher EMG‐DOT scores correlated with better patient experience survey scores (r = 0.42, P = .047), but not with simulated patient or AANEM self‐assessment examination scores.DiscussionEnd‐of‐rotation evaluations can provide valid assessments of trainee performance when completed by individuals with ample opportunities to directly observe trainees. Inclusion of observational assessments by technicians and patients provides a more comprehensive view of trainee performance. Workplace‐ and classroom‐based assessments provide complementary information about trainee performance, reflecting underlying differences in types of skills measured.

Abstract Objective Gray scale ultrasound (US) has been demonstrated to be a sensitive and specific tool in the diagnosis of pediatric neuromuscular disease (NMD). With recent advances in genetic testing, the diagnostic work up for NMD has evolved. The purpose of this study was to compare the current diagnostic value of gray scale US to previously defined sensitivities and specificities to determine when this test can add value to a patient's diagnostic workup. Methods Standardized quantitative gray scale US imaging was performed on 148 pediatric patients presenting for electrodiagnostic testing to evaluate for NMD. Patients were categorized as having an NMD, a non‐NMD, or as “uncertain.” The US results were defined as normal, borderline or abnormal based on echointensity values. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the test were calculated. Results Forty‐five percent of the patients had an NMD, 54% a non‐NMD, and in 1% the diagnosis remained uncertain. US was abnormal in 73% of myopathies, 63% of neuromuscular junction disorders, 60% of generalized neuropathies and 58% of focal neuropathies. After excluding patients in whom muscle US was not expected to be abnormal (eg, sensory neuropathy), sensitivity was 83%, specificity 79%, PPV 75%, NPV 86%, and accuracy 81%. Conclusions Quantitative gray scale muscle US still has good diagnostic value as a screening tool in pediatric NMD. As with any diagnostic test, muscle US is best used in conjunction with history and physical examination to increase specificity and diagnostic yield.