Frank, Ryan D.

Abstract INTRODUCTION We describe the reliability of remote self‐administered digital cognitive measures completed via the Mayo Test Drive (MTD) web‐based platform. METHODS A total of 1846 participants (mean age = 70, SD = 12, range 31–101; 48% male; 96% White; 99% non‐Hispanic; 97% cognitively unimpaired) with two to four complete MTD sessions at ∼7.5‐month intervals were included. Test‐retest reliability was assessed using single‐rating, absolute‐agreement, and two‐way mixed intraclass correlation coefficients (ICCs) with 95% confidence intervals. ICCs for in‐person‐administered traditional neuropsychological measures were compared to MTD for a subset of 244 participants. Analyses also compared reliability across demographic, residential, and session context subgroups. RESULTS Reliability was good for the MTD Composite (total ICC = 0.79 [0.77, 0.80]), and moderate‐to‐good for the primary outcome variables for each MTD subtest (total ICCs 0.70–0.83 for Stricker Learning Span and Symbols). The reliability of the remote self‐administered MTD was similar to in‐person‐administered cognitive measures. DISCUSSION MTD showed moderate‐to‐good reliability, supporting its use in longitudinal monitoring.

Abstract Background Self‐administered and remote digital cognitive assessments provide a scalable method for screening and longitudinal cognitive monitoring. Mayo Test Drive (MTD) is one such assessment with previously demonstrated usability, reliability, validity and multi‐device compatibility. MTD includes a computer‐adaptive word list memory test (Stricker Learning Span) and a measure of processing speed/executive functioning (Symbols Test) that combine to provide an MTD Composite. This study aimed to develop a conditional normative model predicting longitudinal cognitive change for MTD, and to validate the model in two independent samples of cognitively unimpaired (CU) and mild cognitive impairment (MCI)/dementia (DEM) participants. Method 1355 CU participants at initial MTD assessment with 2‐4 consecutive MTD assessments every 7.5 months comprised the development dataset. Linear mixed effects models were used to construct a conditional normative model for MTD assessments 2‐4. The baseline model was chosen a priori , consisting of age, sex, education, baseline score, number of assessments, and time. Device type, race, and baseline score by time interactions were also considered. Variables were only added if a >1% increment in r‐squared was observed. The models were externally validated in 113 CU and 51 MCI/DEM patients. Following model development, the data were updated and additional follow‐up MTD assessments were added for many participants, including the addition of additional CU validation data at test 3 ( N = 126) and 4 ( N = 279). T‐tests compared observed and predicted differences and residual z‐scores between development and validation at each follow‐up assessment. Result Table 1 presents demographic variables. No additional variables increased the r‐squared by >1%. The final models are presented in Table 2. Age, baseline score, sex, and education were significant. Residual z‐scores were not significantly different between CU development and CU validation cohorts, indicating observed scores were in line with model predicted scores. Residual z‐scores were significantly worse among MCI/DEM compared to CU development in 11/12 comparisons (Figure 1), indicating observed scores were below model predicted expectations and implying sensitivity to longitudinal cognitive decline. Conclusion These longitudinal normative models can be used to identify individuals with follow‐up scores outside of expected predicted ranges, signifying abnormal longitudinal cognitive decline.

AbstractBackgroundMayo Test DRIVE (MTD): Test Development through Rapid Iteration, Validation and Expansion, is a web‐based multi‐device (smartphone, tablet, personal computer) platform optimized for remote self‐administered cognitive assessment that includes a computer‐adaptive word list memory test (Stricker Learning Span; SLS), a measure of processing speed/executive functioning (Symbols Test) and a score combining both the memory and processing speed/executive functioning measures (MTD Composite). We aimed to determine the reliability of MTD measures across follow‐up sessions.MethodIndividuals participating in this ancillary study who completed at least two and up to four complete MTD sessions were included, resulting in 1,293 participants (mean age = 70.6, SD = 11.6; mean education = 15.6, SD = 2.3; 48.6% male; 96.9% White; 99.1% non‐Hispanic; 97.1% cognitively unimpaired) from parent studies (see Table 1). MTD raw scores were winsorized at 1% and 99% to minimize the impact of outliers. Test‐retest reliability was assessed using single‐rating, absolute‐agreement, and two‐way mixed ICCs and results are described using descriptives from Koo et al. (2016). ICCs for in‐person‐administered traditional neuropsychological measures are also reported alongside MTD for a subset of participants newly enrolled into the MCSA (to ensure participants were test naïve to both MTD and in‐person tests).ResultReliability was good for the MTD Composite raw [total ICC = 0.78; see Table 2 for session‐to‐session ICCs and 95% CIs], and moderate‐to‐good for the primary outcome variables for each subtest [total ICC = 0.74 for both SLS and Symbols]. Weighting by accuracy negatively impacted reliability of Symbols and the Composite; without this weighting, MTD Composite and Symbols variables showed higher reliability (MTD Composite z total ICC = 0.80; Symbols response time total ICC = 0.84; see Table 2). Alternative methods of defining response time on Symbols showed nearly identical reliability values. SLS sum of trials (primary variable) had subtly higher reliability than separate measures of learning (1‐5 correct, max span) or delay. Reliability of remote MTD and in‐person‐administered measures and select comparisons are provided in Table 3.ConclusionMTD showed moderate to good reliability, similar to in‐person‐administered traditional neuropsychological tests. Future work will examine the effect of demographics, device type, and session interference on reliability.

AbstractBackgroundFew normative data for computerized measures administered in unsupervised remote environments are available. We aimed to determine what variables to include in normative models for remote self‐administered assessments, develop normative data for measures administered through Mayo Test Drive (MTD, a multi‐device remote cognitive assessment platform) and evaluate application of norms.Method1240 adults ages 33‐100 (96% White) from the Mayo Clinic Study of Aging and Mayo Alzheimer’s Disease Research Center met normative sample inclusion criteria that included a concordant Cognitively Unimpaired (CU) diagnosis (3 independent raters all diagnosed CU) and CDR = 0 (see Table 1 for sample characteristics). Raw test scores were converted to normalized scaled scores. We derived regression‐based normative data that adjusted for age, age2, sex and education (base model); alternative models are also provided (age+age2+sex; age+ age2). We evaluated whether additional model terms beyond the base model were needed using an a priori cut‐off of 1% variance improvement in adjusted R2 over and above the base model. We examined rates of low test performance defined as performance below ‐1 SD in independent validation samples (n = 167 concordant CU, n = 64 mild cognitive impairment (MCI), n = 14 dementia). Rates are considered significantly different than expected when 95% confidence intervals (CIs) do not include the expected 14.7% base rate frequency.ResultNo additional potential model terms met our a priori cut‐off for inclusion beyond the base model (see Table 2), including other quadratic and cubic terms or interactions, device type, response input type, potential session interference, and national area deprivation index (ADI available in subset; N = 776). Application of norms showed expected rates of low test performance among CU participants (95% CI includes expected base rate) and significantly greater than expected rates of low test performance for all variables in MCI (e.g., 62.5% for composite) and dementia (e.g., 71.4% for composite; see Table 3).ConclusionTypical normative models appear appropriate for remote self‐administered measures and are sensitive to cognitive impairment. Device type and response input did not explain enough variance to warrant inclusion in normative models but are important for individual level interpretation. Future work will improve representation of individuals from under‐represented groups.

AbstractBackgroundFew normative data for computerized measures administered in unsupervised remote environments are available. We aimed to determine what variables to include in normative models for remote self‐administered assessments, develop normative data for measures administered through Mayo Test Drive (MTD, a multi‐device remote cognitive assessment platform) and evaluate application of norms.Method1240 adults ages 33‐100 (96% White) from the Mayo Clinic Study of Aging and Mayo Alzheimer’s Disease Research Center met normative sample inclusion criteria that included a concordant Cognitively Unimpaired (CU) diagnosis (3 independent raters all diagnosed CU) and CDR = 0 (see Table 1 for sample characteristics). Raw test scores were converted to normalized scaled scores. We derived regression‐based normative data that adjusted for age, age2, sex and education (base model); alternative models are also provided (age+age2+sex; age+ age2). We evaluated whether additional model terms beyond the base model were needed using an a priori cut‐off of 1% variance improvement in adjusted R2 over and above the base model. We examined rates of low test performance defined as performance below ‐1 SD in independent validation samples (n = 167 concordant CU, n = 64 mild cognitive impairment (MCI), n = 14 dementia). Rates are considered significantly different than expected when 95% confidence intervals (CIs) do not include the expected 14.7% base rate frequency.ResultNo additional potential model terms met our a priori cut‐off for inclusion beyond the base model (see Table 2), including other quadratic and cubic terms or interactions, device type, response input type, potential session interference, and national area deprivation index (ADI available in subset; N = 776). Application of norms showed expected rates of low test performance among CU participants (95% CI includes expected base rate) and significantly greater than expected rates of low test performance for all variables in MCI (e.g., 62.5% for composite) and dementia (e.g., 71.4% for composite; see Table 3).ConclusionTypical normative models appear appropriate for remote self‐administered measures and are sensitive to cognitive impairment. Device type and response input did not explain enough variance to warrant inclusion in normative models but are important for individual level interpretation. Future work will improve representation of individuals from under‐represented groups.

Background: Conventional normative samples include individuals with undetected Alzheimer’s disease neuropathology, lowering test sensitivity for cognitive impairment. Objective: We developed Mayo Normative Studies (MNS) norms limited to individuals without elevated amyloid or neurodegeneration (A–N–) for Rey’s Auditory Verbal Learning Test (AVLT). We compared these MNS A–N– norms in female, male, and total samples to conventional MNS norms with varying levels of demographic adjustments. Methods: The A–N– sample included 1,059 Mayo Clinic Study of Aging cognitively unimpaired (CU) participants living in Olmsted County, MN, who are predominantly non-Hispanic White. Using a regression-based approach correcting for age, sex, and education, we derived fully-adjusted T-score formulas for AVLT variables. We validated these A–N– norms in two independent samples of CU ( n = 261) and mild cognitive impairment (MCI)/dementia participants ( n = 392) > 55 years of age. Results: Variability associated with age decreased by almost half in the A–N– norm sample relative to the conventional norm sample. Fully-adjusted MNS A–N– norms showed approximately 7– 9% higher sensitivity to MCI/dementia compared to fully-adjusted MNS conventional norms for trials 1– 5 total and sum of trials. Among women, sensitivity to MCI/dementia increased with each normative data refinement. In contrast, age-adjusted conventional MNS norms showed greatest sensitivity to MCI/dementia in men. Conclusions: A–N– norms show some benefits over conventional normative approaches to MCI/dementia sensitivity, especially for women. We recommend using these MNS A–N– norms alongside MNS conventional norms. Future work is needed to determine if normative samples that are not well characterized clinically show greater benefit from biomarker-refined approaches.

AbstractINTRODUCTIONPremenopausal bilateral oophorectomy (PBO) is associated with later‐life cognition, but the underlying brain changes remain unclear. We assessed the impact of PBO and PBO age on white matter integrity.METHODSFemale participants with regional diffusion tensor imaging (DTI) metrics of fractional anisotropy (FA) and mean diffusivity (MD) were included (22 with PBO < 40 years; 43 with PBO 40‐45 years; 39 with PBO 46‐49 years; 907 referents without PBO < 50 years). Linear regression models adjusted for age and apolipoprotein E (APOE) genotype.RESULTSFemales with PBO < 40 years, compared to referents, had lower FA and higher MD in the anterior corona radiata, genu of the corpus collosum, inferior fronto‐occipital fasciculus, superior occipital, and superior temporal white matter. Females who underwent PBO between 45 and 49 also had some changes in white matter integrity.DISCUSSIONFemales who underwent PBO < 40 years had reduced white matter integrity across multiple regions in later‐life. These results are important for females considering PBO for noncancerous conditions.Highlights Females with premenopausal bilateral oophorectomy (PBO) < 40 years had lower FA versus referents. Females with PBO < 40 years had higher MD in many regions versus referents. Adjusting for estrogen replacement therapy use did not attenuate results. Females with PBO 45‐49 years also had some white matter changes versus referents.