Westover, M. Brandon

Abstract Objective To measure the relative levels of signal and noise in expert diagnosis of epilepsy. Methods Twenty multinational epileptologists independently reviewed 50 vignettes of adult and pediatric patients presenting with suspected seizure(s) on two separate occasions with a ≥30‐day washout period. Experts provided a diagnosis of epilepsy or non‐epilepsy based on clinical information and, if requested, routine EEG and neuroimaging data. Cases had an established clinical diagnosis of epilepsy or non‐epilepsy based on capture of habitual paroxysmal events on video‐EEG or long‐term clinical follow‐up. Experts' judgments were analyzed to decompose variability into different sources: signal (objective differences between cases), level noise (experts' bias toward over/under‐diagnosis), pattern noise (experts' idiosyncratic reactions to specific case features), and occasion noise (inconsistency across occasions). Results The probability of an expert making a different diagnosis for a given case on two different occasions was 16%. The probability of two different experts making a different diagnosis for the same case was 26%. Signal (case “difficulty”) accounted for 66–69% of total variation, with 31–34% attributable to noise. Level noise was the largest contributor in the absence of EEG/neuroimaging results (23%), while pattern noise dominated when test results were available (24%). Occasion noise contributed relatively little (1%) but was still sufficient to cause diagnostic reversals in 16–22% between occasions. Significance The degree of noise in expert diagnosis of epilepsy is substantial, stemming primarily from physicians' idiosyncratic interpretations of case features and variable dispositions toward over‐ or under‐diagnosis. Strategies to improve reliability are needed, including standardized data collection protocols and structured decision algorithms. For “difficult cases,” where expert reliability and accuracy are lowest, our findings support current clinical practice which favors early referral for video‐EEG monitoring over reliance on diagnostic anchoring. This diagnostic pathway may become more accessible with advances in EEG technology (e.g., wearable devices) and artificial intelligence.

Purpose: Delivering optimal care to patients with seizures and epilepsy requires all EEGs to be interpreted accurately and reliably. This study investigated neurology professionals' opinions on the ideal standards for EEG in clinical care. Methods: We developed an anonymous e-survey targeting practicing and trainee neurologists focused on participants' demographics, clinical practice characteristics, and views on optimal EEG standards of care—including whether an EEG certification test is needed and whether postresidency/fellowship training in EEG/epilepsy is necessary for neurologists who interpret outpatient/routine EEGs in practice. The survey was hosted by the Neurology Clinical Practice—Practice Current, and it was distributed online through the American Academy of Neurology, American Epilepsy Society, American Clinical Neurophysiology Society, and International League Against Epilepsy, and through social media. Results: Two hundred eighty-three responses were included: 119 from EEG/epilepsy-trained neurologists, 83 from non-EEG/epilepsy-trained neurologists, 75 from trainees, and 6 from advanced care providers. Most participants (78%) agreed that “an objective certification test of ability to interpret EEGs is needed for all those who interpret EEGs in clinical practice.” Most participants (71%) believed that outpatient/routine EEGs may be read only by neurologists with EEG/epilepsy training; this opinion was more prevalent among EEG/epilepsy-trained (83%) versus non-EEG/epilepsy-trained neurologists (55%). Conclusions: Our neurology community should discuss the need to develop and implement a certification test of ability for all neurologists who wish to interpret EEGs in clinical practice. In addition, it is imperative to improve in-residency EEG education to ensure that neurology graduates achieve EEG competence before entering the workforce.

AbstractStudies suggest that self‐reported seizure diaries suffer from 50% under‐reporting on average. It is unknown to what extent this impacts medication management. This study used simulation to predict the seizure outcomes of a large heterogeneous clinic population treated with a standardized algorithm based on self‐reported seizures. Using CHOCOLATES, a state‐of‐the‐art realistic seizure diary simulator, 100 000 patients were simulated over 10 years. A standard algorithm for medication management was employed at 3 month intervals for all patients. The impact on true seizure rates, expected seizure rates, and time‐to‐steady‐dose were computed for self‐reporting sensitivities 0%–100%. Time‐to‐steady‐dose and medication use mostly did not depend on sensitivity. True seizure rate decreased minimally with increasing self‐reporting in a non‐linear fashion, with the largest decreases at low sensitivity rates (0%–10%). This study suggests that an extremely wide range of sensitivity will have similar seizure outcomes when patients are clinically treated using an algorithm similar to the one presented. Conversely, patients with sensitivity ≤10% would be expected to benefit (via lower seizure rates) from objective devices that provide even small improvements in seizure sensitivity.

AbstractObjectiveEpilepsy management employs self‐reported seizure diaries, despite evidence of seizure underreporting. Wearable and implantable seizure detection devices are now becoming more widely available. There are no clear guidelines about what levels of accuracy are sufficient. This study aimed to simulate clinical use cases and identify the necessary level of accuracy for each.MethodsUsing a realistic seizure simulator (CHOCOLATES), a ground truth was produced, which was then sampled to generate signals from simulated seizure detectors of various capabilities. Five use cases were evaluated: (1) randomized clinical trials (RCTs), (2) medication adjustment in clinic, (3) injury prevention, (4) sudden unexpected death in epilepsy (SUDEP) prevention, and (5) treatment of seizure clusters. We considered sensitivity (0%–100%), false alarm rate (FAR; 0–2/day), and device type (external wearable vs. implant) in each scenario.ResultsThe RCT case was efficient for a wide range of wearable parameters, though implantable devices were preferred. Lower accuracy wearables resulted in subtle changes in the distribution of patients enrolled in RCTs, and therefore higher sensitivity and lower FAR values were preferred. In the clinic case, a wide range of sensitivity, FAR, and device type yielded similar results. For injury prevention, SUDEP prevention, and seizure cluster treatment, each scenario required high sensitivity and yet was minimally influenced by FAR.SignificanceThe choice of use case is paramount in determining acceptable accuracy levels for a wearable seizure detection device. We offer simulation results for determining and verifying utility for specific use case and specific wearable parameters.