Benbadis, Selim R.

Abstract Objective To measure the relative levels of signal and noise in expert diagnosis of epilepsy. Methods Twenty multinational epileptologists independently reviewed 50 vignettes of adult and pediatric patients presenting with suspected seizure(s) on two separate occasions with a ≥30‐day washout period. Experts provided a diagnosis of epilepsy or non‐epilepsy based on clinical information and, if requested, routine EEG and neuroimaging data. Cases had an established clinical diagnosis of epilepsy or non‐epilepsy based on capture of habitual paroxysmal events on video‐EEG or long‐term clinical follow‐up. Experts' judgments were analyzed to decompose variability into different sources: signal (objective differences between cases), level noise (experts' bias toward over/under‐diagnosis), pattern noise (experts' idiosyncratic reactions to specific case features), and occasion noise (inconsistency across occasions). Results The probability of an expert making a different diagnosis for a given case on two different occasions was 16%. The probability of two different experts making a different diagnosis for the same case was 26%. Signal (case “difficulty”) accounted for 66–69% of total variation, with 31–34% attributable to noise. Level noise was the largest contributor in the absence of EEG/neuroimaging results (23%), while pattern noise dominated when test results were available (24%). Occasion noise contributed relatively little (1%) but was still sufficient to cause diagnostic reversals in 16–22% between occasions. Significance The degree of noise in expert diagnosis of epilepsy is substantial, stemming primarily from physicians' idiosyncratic interpretations of case features and variable dispositions toward over‐ or under‐diagnosis. Strategies to improve reliability are needed, including standardized data collection protocols and structured decision algorithms. For “difficult cases,” where expert reliability and accuracy are lowest, our findings support current clinical practice which favors early referral for video‐EEG monitoring over reliance on diagnostic anchoring. This diagnostic pathway may become more accessible with advances in EEG technology (e.g., wearable devices) and artificial intelligence.

Abstract Background and purpose The aim of this study was to evaluate the quality of smartphone videos (SVs) of neurologic events in adult epilepsy outpatients. The use of home video recording in patients with neurological disease states is increasing. Experts interpretation of outpatient smartphone videos of seizures and neurological events has demonstrated similar diagnostic accuracy to inpatient video‐electroencephalography (EEG) monitoring. Methods A prospective, multicenter cohort study was conducted to evaluate SV quality in patients with paroxysmal neurologic events from August 15, 2015 through August 31, 2018. Epileptic seizures (ESs), psychogenic nonepileptic attacks (PNEAs), and physiologic nonepileptic events (PhysNEEs) were confirmed by video‐EEG monitoring. Experts and senior neurology residents blindly viewed cloud‐based SVs without clinical information. Quality ratings with regard to technical and operator‐driven metrics were provided in responses to a survey. Results Forty‐four patients (31 women, age 45.1 years [ r = 20–82]) were included and 530 SVs were viewed by a mean of seven experts and six residents; one video per patient was reviewed for a mean of 133.8 s ( r = 9–543). In all, 30 patients had PNEAs, 11 had ESs, and three had PhysNEEs. Quality was suitable in 70.8% of SVs (375/530 total views), with 36/44 (81.8%) patient SVs rated as adequate by the majority of reviewers. Accuracy improved with the presence of convulsive features from 72.4% to 98.2% in ESs and from 71.1% to 95.7% in PNEAs. An accurate diagnosis was given by all reviewers (100%) in 11/44 SVs (all PNEAs). Audio was rated as good by 86.2% of reviewers for these SVs compared with 75.4% for the remaining SVs ( p = 0.01). Lighting was better in SVs associated with high accuracy ( p = 0.06), but clarity was not ( p = 0.59). Poor video quality yielded unknown diagnoses in 24.2% of the SVs reviewed. Features hindering diagnosis were limited interactivity, restricted field of view and short video duration. Conclusions Smartphone video quality is adequate for clinical interpretation in the majority of patients with paroxysmal neurologic events. Quality can be optimized by encouraging interactivity with the patient, adequate duration of the SV, and enlarged field of view during videography. Quality limitations were primarily operational though accuracy remained for SV review of ESs and PNEAs.