Kennedy, Richard E

Abstract Background Delirium is often underdiagnosed in clinical practice and is not routinely coded for billing. While manual chart review can identify delirium, it is labor-intensive and impractical for large-scale studies. Natural language processing (NLP) can analyze unstructured text in electronic health records (EHRs) to extract meaningful clinical information. Methods To support national integration of NLP for EHR-based delirium identification across different institutions, we launched the Delirium Interest Group within the national Evolve to Next-Gen Accrual to Clinical Trials (ENACT) NLP Working Group. This paper outlines our initial efforts to standardize, evaluate, and translate an NLP-based delirium detection model into the i2b2/ENACT platform. Results Multisite contextual inquiry identified several key challenges, including variations in local screening practices (eg, tools used, documentation frequency, and quality control), the need for harmonized definitions in the context of EHRs, and the complexity of modeling temporal logic. Multisite NLP evaluation revealed variable performance degradation driven by differences in delirium screening practices, clinical documentation patterns and semantics, and note syntactic structures. Conclusion Our work represents an important first step toward enabling scalable and standardized NLP-based delirium detection across institutions. By engaging diverse institutions through the ENACT NLP Working Group, we identified shared challenges and site-specific variations that impact model implementation and performance. Our collaborative approach enabled the development of a more robust framework for delirium identification across heterogeneous EHR systems. Future efforts will build on this foundation to enhance the validity, usability, and translational impact of delirium detection.

Abstract Objective: Electronic Health Record (EHR) data are critical for advancing translational research and AI technologies. The ENACT network offers access to structured EHR data across 57 CTSA hubs. However, substantial information is contained in clinical narratives, requiring natural language processing (NLP) for research. The ENACT NLP Working Group was formed to make NLP-derived clinical information accessible and queryable across the network. Methods: We established the ENACT NLP Working Group with 13 sites selected based on criteria including clinical notes access, IT infrastructure, NLP expertise, and institutional support. We divided sites into five focus groups targeting clinical tasks within disease contexts. Each focus group consisted of two development sites and two validation sites. We extended the ENACT ontology to standardize NLP-derived data and conducted multisite evaluations using the Open Health Natural Language Processing (OHNLP) Toolkit. Results: The working group achieved 100% site retention and deployed NLP infrastructure across all sites. We developed and validated NLP algorithms for rare disease phenotyping, social determinants of health, opioid use disorder, sleep phenotyping, and delirium phenotyping. Performance varied across sites (F1 scores 0.53–0.96), highlighting data heterogeneity impacts. We extended the ENACT common data model and ontology to incorporate NLP-derived data while maintaining Shared Health Research Informatics NEtwork (SHRINE) compatibility. Conclusion: This demonstrates feasibility of deploying NLP infrastructure across large, federated networks. The focus group approach proved more practical than general-purpose approaches. Key lessons include the challenge of data heterogeneity and importance of collaborative governance. This work also provides a foundation that other networks can build on to implement NLP capabilities for translational research.

Importance The Clinical Dementia Rating (CDR) is a well-validated instrument widely used to detect and stage dementia due to Alzheimer disease. The digital Electronic Clinical Dementia Rating (eCDR) can be remotely self-administered and automatically scored, with potential to facilitate efficient dementia screening and staging. Objective To evaluate the association of the eCDR with the CDR and other in-clinic assessments for screening older adults for cognitive impairment. Design, Setting, and Participants This multisite, cross-sectional study used baseline data from a longitudinal, observational study from 2020 to 2023, including up to 3 years of follow-up. Participants were enrolled from 3 Alzheimer Disease Research Centers and the Brain Health Registry. Participants (aged ≥55 years, with a study partner, and no acute or unstable major medical conditions) were recruited during in-clinic visits or by automated emails. Exposures Participants completed the Uniform Data Set Version 3 (UDS; including the CDR) in supervised clinical research settings, and then completed the eCDR remotely, online and unsupervised, using their own device. Main Outcomes and Measures The primary outcomes were eCDR scores (item; categorical box and global; continuous box and global), CDR scores (item; categorical box and global), and UDS assessment scores. Associations were evaluated using linear and logistic regressions. Results A total of 3565 participants were contacted, and 288 were enrolled. Among 173 participants with item-level data (mean [SD] age, 70.84 [7.65] years; 76 women [43.9%]), eCDR to CDR concordance was 90% or higher for 33 items (63%) and 70% to 89% for 13 items (25%). Box (domain) level concordance ranged from 80% (memory) to 99% (personal care). The global score concordance rate was 81%. κ statistics were fair to moderate. Among 206 participants with box and global scores (mean [SD] age, 71.34 [7.68] years; 95 women [46.1%]), eCDR continuous global score was associated with CDR global (categorical) score with an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.70-0.87). Correlations between eCDR and in-clinic UDS assessments were similar to those between CDR sum of box scores and the same in-clinic assessments. Conclusions and Relevance These findings suggest that the eCDR is valid and has potential use for screening and assessment of older adults for cognitive and functional decline related to Alzheimer disease. Instrument optimization and validation in diverse cohorts in remote settings are crucial for evaluating scalability and eCDR utility in clinical research, trials, and health care settings.

Abstract Objective The Financial Capacity Instrument-Short Form (FCI-SF) is a performance-based measure of everyday financial skills that takes 15 min to administer. Although the FCI-SF has demonstrated excellent psychometric properties, advanced psychometric methods such as item response theory (IRT) can provide important information on the performance of individual test items in measuring financial capacity and in distinguishing between healthy and cognitively impaired individuals. Method Participants were 272 older adults diagnosed with mild cognitive impairment (MCI) and 1,344 cognitively healthy controls recruited from the Mayo Clinic Study of Aging at the Mayo Clinic in Rochester, Minnesota and also from the Cognitive Observations in Seniors study at the University of Alabama at Birmingham. Participants in each study were administered the FCI-SF, which evaluates coin/currency calculation, financial conceptual knowledge, use of a checkbook/register, and use of a bank statement. Results A unidimensional two-parameter logistic model best fit the 37 FCI-SF Test items, and most FCI-SF items fit the unidimensional two-parameter model well. The results indicated that all FCI-SF items robustly distinguished cognitively healthy controls from persons with MCI. Conclusions The study results showed that the FCI-SF performed well under IRT analysis, further highlighted the psychometric properties of the FCI-SF as a valid and reliable measure of financial capacity, and demonstrated the clinical utility of the FCI-SF in distinguishing between cognitively normal and cognitively impaired individuals.

AbstractIntroductionTo address the need for remote assessments of cognitive decline and dementia, we developed and administered electronic versions of the Clinical Dementia Rating (CDR®) and the Financial Capacity Instrument‐Short Form (FCI‐SF) (F‐CAP®), called the eCDR and eFCI, respectively.MethodsThe CDR and FCI‐SF were adapted for remote, unsupervised, online use based on item response analysis of the standard instruments. Participants completed the eCDR and eFCI first in clinic, and then at home within 2 weeks.ResultsOf the 243 enrolled participants, 179 (73%) cognitively unimpaired (CU), 50 (21%) with mild cognitive impairment (MCI) or dementia, and 14 (6%) with an unknown diagnosis, 84% and 85% of them successfully completed the eCDR and eFCI, respectively, at home.DiscussionThese results show initial feasibility in developing and administering online instruments to remotely assess and monitor cognitive decline along the CU to MCI/very mild dementia continuum. Validation is an important next step.

Background Censoring that is dependent on covariates associated with survival can arise in randomized trials due to changes in recruitment and eligibility criteria to minimize withdrawals, potentially leading to biased treatment effect estimates. Imputation approaches have been proposed to address censoring in survival analysis; while these approaches may provide unbiased estimates of treatment effects, imputation of a large number of outcomes may over- or underestimate the associated variance based on the imputation pool selected. Purpose We propose an improved method, risk-stratified imputation, as an alternative to address withdrawal related to the risk of events in the context of time-to-event analyses. Methods Our algorithm performs imputation from a pool of replacement subjects with similar values of both treatment and covariate(s) of interest, that is, from a risk-stratified sample. This stratification prior to imputation addresses the requirement of time-to-event analysis that censored observations are representative of all other observations in the risk group with similar exposure variables. We compared our risk-stratified imputation to case deletion and bootstrap imputation in a simulated dataset in which the covariate of interest (study withdrawal) was related to treatment. A motivating example from a recent clinical trial is also presented to demonstrate the utility of our method. Results In our simulations, risk-stratified imputation gives estimates of treatment effect comparable to bootstrap and auxiliary variable imputation while avoiding inaccuracies of the latter two in estimating the associated variance. Similar results were obtained in analysis of clinical trial data. Limitations Risk-stratified imputation has little advantage over other imputation methods when covariates of interest are not related to treatment. Risk-stratified imputation is intended for categorical covariates and may be sensitive to the width of the matching window if continuous covariates are used. Conclusions The use of the risk-stratified imputation should facilitate the analysis of many clinical trials, in which one group has a higher withdrawal rate that is related to treatment.