Citations

Abstract Background Keratoconus (KC) is a multifactorial disorder with unclear etiology, characterized by localized thinning and a cone-like protrusion of the cornea. The complex etiology of KC exacerbates the lack of an animal model. Previous studies by Tachibana et al. (2002) described an inbred mouse strain (SKC) with a spontaneous, androgen-dependent, cone-like corneal morphology. This study aimed to investigate the corneal phenotypes of SKC mice through an in-depth ophthalmic examination. Methods Mice (n=53) were examined via slit lamp biomicroscopy with fluorescein staining. Spectral-domain optical coherence tomography (SD-OCT) enabled central corneal thickness (CCT) measurement in selected mice (n=26 eyes), and OCT-based pachymetry mapping (n=16 eyes). In vivo corneal confocal microscopy was conducted on eyes to assess cellular morphology (n= 9 eyes). Eyes were collected for histology analysis (n=22). Results Lesions and epithelial breaks were present in ∼95% of eyes (n=101). Neovascularization, perforation, scarring, and hydrops were seen primarily in males. An opaque, unilateral cone-like morphology was exclusive to males (n=11). Male and female corneas showed no significant difference in CCT, though pachymetry mapping revealed regional thinning patterns in both sexes. Loosened epithelial tight junctions, stromal fibrosis, vascularization, and inflammation of variable severity were identified in both sexes. Conclusion This study identified previously unreported corneal phenotypes in SKC mice through ophthalmic examination. Unlike previous studies, gross and histological abnormalities were observed in female SKC mice. Our findings suggest a lower penetrance of the cone-like phenotype (∼20%) than previously reported (∼33%) and support that the conical phenotype in male mice may be secondary to keratitis.

Abstract Background Integrating advanced artificial intelligence (AI) into clinical decision-support often requires the sharing of sensitive patient data with external services, raising privacy concerns. Homomorphic encryption (HE) allows computing directly on encrypted data, without revealing the underlying patient information. Objectives To develop a large language model (LLM)-assisted diagnosis framework while preserving patient privacy in the clinical text analysis, by leveraging HE and using rare disease (RD) diagnosis as a representative application. To demonstrate HE does not hinder the system performance. Materials and Methods Texts from patient histories and a RD knowledge base were embedded by LLMs into vectors, then encrypted using HE to obscure private information while retaining the semantic nuances. Diagnostic recommendations were generated by computing and ranking the similarities between the patient history and RD vectors in the encrypted space. The system was evaluated using 50 synthetic case reports (5 RDs, each with 10 reports). Results Applying HE did protect private information from reverse-embedding attacks. HE imposed little disruption to the diagnostic accuracy, with normalized discounted cumulative gains (nDCG) of 0.6108 ± 0.3412 (encrypted) versus 0.6083 ± 0.3415 (unencrypted). The accuracy and computational performance were tunable and consistent, as demonstrated across five different LLMs. Discussion Our privacy-preserving framework opens tremendous opportunities toward hosting and serving powerful AI solutions across institution boundaries, which would remove the need for local deidentification and incentivize users to access secure external decision-support services. Conclusions Integrating HE with LLM retrieval can promote the dissemination of nonredundant, high-capacity AI services by preserving both privacy and accuracy.

Abstract Background Delirium is often underdiagnosed in clinical practice and is not routinely coded for billing. While manual chart review can identify delirium, it is labor-intensive and impractical for large-scale studies. Natural language processing (NLP) can analyze unstructured text in electronic health records (EHRs) to extract meaningful clinical information. Methods To support national integration of NLP for EHR-based delirium identification across different institutions, we launched the Delirium Interest Group within the national Evolve to Next-Gen Accrual to Clinical Trials (ENACT) NLP Working Group. This paper outlines our initial efforts to standardize, evaluate, and translate an NLP-based delirium detection model into the i2b2/ENACT platform. Results Multisite contextual inquiry identified several key challenges, including variations in local screening practices (eg, tools used, documentation frequency, and quality control), the need for harmonized definitions in the context of EHRs, and the complexity of modeling temporal logic. Multisite NLP evaluation revealed variable performance degradation driven by differences in delirium screening practices, clinical documentation patterns and semantics, and note syntactic structures. Conclusion Our work represents an important first step toward enabling scalable and standardized NLP-based delirium detection across institutions. By engaging diverse institutions through the ENACT NLP Working Group, we identified shared challenges and site-specific variations that impact model implementation and performance. Our collaborative approach enabled the development of a more robust framework for delirium identification across heterogeneous EHR systems. Future efforts will build on this foundation to enhance the validity, usability, and translational impact of delirium detection.

Abstract Error analysis is a critical step in evaluating and improving clinical concept extraction models, the most common clinical natural language processing (NLP) task. Unlike corpus annotation, which follows standardized protocols for creating gold-standard datasets, error analysis requires nuanced judgment grounded in both clinical expertise and NLP knowledge. This is especially important given the heterogeneity of clinical text, where variations in documentation style, note structure, and terminology can substantially influence model behavior. Despite its importance, there is currently no standardized, user-level framework to support systematic error analysis in clinical concept extraction task. In this study, we developed and validated MedError, a machine-assisted, human-in-the-loop framework designed to standardize and enhance error analysis for clinical concept extraction tasks. We collected and manually curated a corpus of 1,187 unique errors from a total of 4,237 notes across three different distinct hospitals. The error categories were defined using our previously validated error taxonomy and included 480 false negatives and 707 false positives across 25 error types and 48 clinical concept categories. We evaluated the performance of three proprietary and three open-source large language models (LLMs) in automatically classifying these errors into 26 and 15 predefined categories. We further developed a machine-assisted framework, MedError, which integrates best practices in error analysis, LLM-assisted classification and reasoning, and a user-friendly interface to enable more efficient, reproducible, and context-aware error analysis. The framework supports both single-site and federated multisite error analysis, facilitating the effective translation of clinical NLP systems into real-world settings.

Abstract Current discussion surrounding the clinical capabilities of generative language models (GLMs) predominantly center around multiple-choice question-answer (MCQA) benchmarks derived from clinical licensing examinations. While accepted for human examinees, characteristics unique to GLMs bring into question the validity of such benchmarks. Here, we validate four benchmarks using eight GLMs, ablating for parameter size and reasoning capabilities, validating via prompt permutation three key assumptions that underpin the generalizability of MCQA-based assessments: that knowledge is applied, not memorized, that semantic consistency will lead to consistent answers, and that situations with no answers can be recognized. While large models are more resilient to our perturbations compared to small models, we globally invalidate these assumptions, with implications for reasoning models. Additionally, despite retaining the knowledge, small models are prone to answer memorization. All models exhibit significant failure in null-answer scenarios. We then suggest several adaptations for more robust benchmark designs more reflective of real-world conditions.

Abstract Objective: Electronic Health Record (EHR) data are critical for advancing translational research and AI technologies. The ENACT network offers access to structured EHR data across 57 CTSA hubs. However, substantial information is contained in clinical narratives, requiring natural language processing (NLP) for research. The ENACT NLP Working Group was formed to make NLP-derived clinical information accessible and queryable across the network. Methods: We established the ENACT NLP Working Group with 13 sites selected based on criteria including clinical notes access, IT infrastructure, NLP expertise, and institutional support. We divided sites into five focus groups targeting clinical tasks within disease contexts. Each focus group consisted of two development sites and two validation sites. We extended the ENACT ontology to standardize NLP-derived data and conducted multisite evaluations using the Open Health Natural Language Processing (OHNLP) Toolkit. Results: The working group achieved 100% site retention and deployed NLP infrastructure across all sites. We developed and validated NLP algorithms for rare disease phenotyping, social determinants of health, opioid use disorder, sleep phenotyping, and delirium phenotyping. Performance varied across sites (F1 scores 0.53–0.96), highlighting data heterogeneity impacts. We extended the ENACT common data model and ontology to incorporate NLP-derived data while maintaining Shared Health Research Informatics NEtwork (SHRINE) compatibility. Conclusion: This demonstrates feasibility of deploying NLP infrastructure across large, federated networks. The focus group approach proved more practical than general-purpose approaches. Key lessons include the challenge of data heterogeneity and importance of collaborative governance. This work also provides a foundation that other networks can build on to implement NLP capabilities for translational research.

BACKGROUND: The impact of covert cerebrovascular disease on falls in the general population is not well-known. Here, we determine the time to a first fall following incidentally detected covert cerebrovascular disease during a clinical neuroimaging episode. METHODS: This longitudinal cohort study assessed computed tomography (CT) and magnetic resonance imaging from 2009 to 2019 of patients aged >50 years registered with Kaiser Permanente Southern California which is a healthcare organization combining health plan coverage with coordinated medical services, excluding those with before stroke/dementia. We extracted evidence of incidental covert brain infarcts (CBI) and white matter hyperintensities/hypoattenuation (WMH) from imaging reports using natural language processing. We examined associations of CBI and WMH with falls requiring medical attention, using Cox proportional hazards regression models with adjustment for 12 variables including age, sex, ethnicity multimorbidity, polypharmacy, and incontinence. RESULTS: We assessed 241 050 patients, mean age 64.9 (SD, 10.42) years, 61.3% female, detecting covert cerebrovascular disease in 31.1% over a mean follow-up duration of 3.04 years. A recorded fall occurred in 21.2% (51 239/241 050) during follow-up. On CT, single fall incidence rate/1000 person-years (p-y) was highest in individuals with both CBI and WMH on CT (129.3 falls/1000 p-y [95% CI, 123.4–135.5]), followed by WMH (109.9 falls/1000 p-y [108.0–111.9]). On magnetic resonance imaging, the incidence rate was the highest with both CBI and WMH (76.3 falls/1000 p-y [95% CI, 69.7–83.2]), followed by CBI (71.4 falls/1000 p-y [95% CI, 65.9–77.2]). The adjusted hazard ratio for single index fall in individuals with CBI on CT was 1.13 (95% CI, 1.09–1.17); versus magnetic resonance imaging 1.17 (95% CI, 1.08–1.27). On CT, the risk for single index fall incrementally increased for mild (1.37 [95% CI, 1.32–1.43]), moderate (1.57 [95% CI, 1.48–1.67]), or severe WMH (1.57 [95% CI, 1.45–1.70]). On magnetic resonance imaging, index fall risk similarly increased with increasing WMH severity: mild (1.11 [95% CI, 1.07–1.17]), moderate (1.21 [95% CI, 1.13–1.28]), and severe WMH (1.34 [95% CI, 1.22–1.46]). CONCLUSIONS: In a large population with neuroimaging, CBI and WMH are independently associated with greater risks of an index fall. Increasing severities of WMH are associated incrementally with fall risk across imaging modalities.

Background Electronic health records (EHRs) serve as a critical tool for clinical documentation, yet their complexity has raised concerns about readability and usability for healthcare providers and patients. Readability scores, such as the Flesch-Kincaid (FK) Grade Level, offer a quantitative method to assess documentation clarity. In this study, the authors evaluated the impact of an EHR transition on the readability of clinical notes across multiple medical specialties. Methods All clinical notes recorded between 2016 and 2019 at a large healthcare organization were evaluated across specialties. The organization transitioned EHR systems during the study period, in May 2018. FK Grade Level scores were calculated for each clinical consultation note and stratified by specialty and EHR. Results Across 10 medical specialties, 630,246 clinical notes were evaluated. The median FK grade levels from notes composed in the new EHR were similar to those composed in the previous EHR (8.228 vs 8.240; p=0.58). The variation in FK grade level as measured by standard deviation was lower in the new EHR compared with the old EHR (1.405 vs 1.764; p <0.01). Conclusions Calculation of FK grade level of clinical notes is a feasible method to assess clinical note readability. Study results showed that clinical documentation readability may not be strongly associated with underlying EHR. Migrating to a new EHR alone should not be expected to improve readability of clinical notes.