Citations

Abstract Background Delirium is often underdiagnosed in clinical practice and is not routinely coded for billing. While manual chart review can identify delirium, it is labor-intensive and impractical for large-scale studies. Natural language processing (NLP) can analyze unstructured text in electronic health records (EHRs) to extract meaningful clinical information. Methods To support national integration of NLP for EHR-based delirium identification across different institutions, we launched the Delirium Interest Group within the national Evolve to Next-Gen Accrual to Clinical Trials (ENACT) NLP Working Group. This paper outlines our initial efforts to standardize, evaluate, and translate an NLP-based delirium detection model into the i2b2/ENACT platform. Results Multisite contextual inquiry identified several key challenges, including variations in local screening practices (eg, tools used, documentation frequency, and quality control), the need for harmonized definitions in the context of EHRs, and the complexity of modeling temporal logic. Multisite NLP evaluation revealed variable performance degradation driven by differences in delirium screening practices, clinical documentation patterns and semantics, and note syntactic structures. Conclusion Our work represents an important first step toward enabling scalable and standardized NLP-based delirium detection across institutions. By engaging diverse institutions through the ENACT NLP Working Group, we identified shared challenges and site-specific variations that impact model implementation and performance. Our collaborative approach enabled the development of a more robust framework for delirium identification across heterogeneous EHR systems. Future efforts will build on this foundation to enhance the validity, usability, and translational impact of delirium detection.

Abstract Objective: Electronic Health Record (EHR) data are critical for advancing translational research and AI technologies. The ENACT network offers access to structured EHR data across 57 CTSA hubs. However, substantial information is contained in clinical narratives, requiring natural language processing (NLP) for research. The ENACT NLP Working Group was formed to make NLP-derived clinical information accessible and queryable across the network. Methods: We established the ENACT NLP Working Group with 13 sites selected based on criteria including clinical notes access, IT infrastructure, NLP expertise, and institutional support. We divided sites into five focus groups targeting clinical tasks within disease contexts. Each focus group consisted of two development sites and two validation sites. We extended the ENACT ontology to standardize NLP-derived data and conducted multisite evaluations using the Open Health Natural Language Processing (OHNLP) Toolkit. Results: The working group achieved 100% site retention and deployed NLP infrastructure across all sites. We developed and validated NLP algorithms for rare disease phenotyping, social determinants of health, opioid use disorder, sleep phenotyping, and delirium phenotyping. Performance varied across sites (F1 scores 0.53–0.96), highlighting data heterogeneity impacts. We extended the ENACT common data model and ontology to incorporate NLP-derived data while maintaining Shared Health Research Informatics NEtwork (SHRINE) compatibility. Conclusion: This demonstrates feasibility of deploying NLP infrastructure across large, federated networks. The focus group approach proved more practical than general-purpose approaches. Key lessons include the challenge of data heterogeneity and importance of collaborative governance. This work also provides a foundation that other networks can build on to implement NLP capabilities for translational research.

Aim: AI research, development, and implementation are expanding at an exponential pace across healthcare. This paradigm shift in healthcare research has led to increased demands for clinical outcomes, all at the expense of a significant gap in AI literacy within the healthcare field. This has further translated to a lack of tools in creating a framework for literature in the AI in medicine domain. We propose HUMANE (Harmonious Understanding of Machine Learning Analytics Network), a checklist for establishing an international consensus for authors and reviewers involved in research focused on artificial intelligence (AI) or machine learning (ML) in medicine. Methods: This study was conducted using the Delphi method by devising a survey using the Google Forms platform. The survey was developed as a checklist containing 8 sections and 56 questions with a 5-point Likert scale. Results: A total of 33 survey respondents were part of the initial Delphi process with the majority (45%) in the 36–45 years age group. The respondents were located across the USA (61%), UK (24%), and Australia (9%) as the top 3 countries, with a pre-dominant healthcare background (42%) as early-career professionals (3–10 years’ experience) (42%). Feedback showed an overall agreeable consensus (mean ranges 4.1–4.8, out of 5) as cumulative scores throughout all sections. The majority of the consensus was agreeable with the Discussion (Other) section of the checklist (median 4.8 (interquartile range (IQR) 4.8-4.8)), whereas the least agreed section was the Ground Truth (Expert(s) review) section (median 4.1 (IQR 3.9–4.2)) and the Methods (Outcomes) section (median 4.1 (IQR 4.1–4.1)) of the checklist. The final checklist after consensus and revision included a total of 8 sections and 50 questions. Conclusions: The HUMANE international consensus has reflected on further research on the potential of this checklist as an established consensus in improving the reliability and quality of research in this field.

Background Machine learning (ML) is pervasive in all fields of research, from automating tasks to complex decision‐making. However, applications in different specialities are variable and generally limited. Like other conditions, the number of studies employing ML in hypertension research is growing rapidly. In this study, we aimed to survey hypertension research using ML, evaluate the reporting quality, and identify barriers to ML's potential to transform hypertension care. Methods and Results The Harmonious Understanding of Machine Learning Analytics Network survey questionnaire was applied to 63 hypertension‐related ML research articles published between January 2019 and September 2021. The most common research topics were blood pressure prediction (38%), hypertension (22%), cardiovascular outcomes (6%), blood pressure variability (5%), treatment response (5%), and real‐time blood pressure estimation (5%). The reporting quality of the articles was variable. Only 46% of articles described the study population or derivation cohort. Most articles (81%) reported at least 1 performance measure, but only 40% presented any measures of calibration. Compliance with ethics, patient privacy, and data security regulations were mentioned in 30 (48%) of the articles. Only 14% used geographically or temporally distinct validation data sets. Algorithmic bias was not addressed in any of the articles, with only 6 of them acknowledging risk of bias. Conclusions Recent ML research on hypertension is limited to exploratory research and has significant shortcomings in reporting quality, model validation, and algorithmic bias. Our analysis identifies areas for improvement that will help pave the way for the realization of the potential of ML in hypertension and facilitate its adoption.

Abstract Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this field. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases, including Ovid Embase, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, from January 1st, 2012, to January 4th, 2023. The screening process involved multiple reviewers, and we included 184 papers for the final review. Results: We conducted a detailed analysis and discussion of various aspects of clinical IR research, including publication year, data sources, methods, techniques, evaluation metrics, shared tasks, and applications. Our analysis revealed key research areas in clinical IR, such as indexing, ranking, and query expansion, and identified opportunities for future research in these areas.

AbstractIntroductionComputable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine‐interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T.MethodsWe examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject‐predicate‐object triples to more clearly differentiate metadata categories.ResultsWe defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK‐to‐CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories.ConclusionA wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.

A critical goal of multidisciplinary scientific teams is to integrate knowledge from diverse disciplines for the purpose of developing novel insights and innovations. For example, multidisciplinary translational teams (MTTs) which typically include physicians, biologists, statisticians, and informaticians, aim to integrate biological and clinical knowledge leading to innovations for improving health outcomes. However, such teams face numerous barriers in integrating multidisciplinary knowledge, which is further exacerbated by the explosion of molecular and clinical data generated from millions of patients. Here, we explore the use of a visual analytical representation to help MTTs integrate molecular and clinical data with the goal of accelerating translational insights. The results suggest that the visual analytical representation functioned as a “computational evolving boundary object” which (a) evolved through several emergent states that progressively helped integrate diverse disciplinary knowledge, (b) enabled team members to play primary and supportive roles in evolving the data representation resulting in a more egalitarian team structure, and (c) enabled the team to arrive at novel translational insights leading to domain and methodology publications. However, the interventions also revealed limitations in the approach motivating new visual analytical approaches. These results suggest (a) implications for theory related to modeling computational evolving boundary objects (CEBOs) as an instance of team-centered informatics, and (b) implications for practice related to the design and use of interactive features that enable teams to fluidly evolve CEBOs through emergent states, with the goal of deriving novel insights from large multiomics datasets.

AbstractThe recent announcement of the Precision Medicine Initiative by President Obama has brought precision medicine (PM) to the forefront for healthcare providers, researchers, regulators, innovators, and funders alike. As technologies continue to evolve and datasets grow in magnitude, a strong computational infrastructure will be essential to realize PM’s vision of improved healthcare derived from personal data. In addition, informatics research and innovation affords a tremendous opportunity to drive the science underlying PM. The informatics community must lead the development of technologies and methodologies that will increase the discovery and application of biomedical knowledge through close collaboration between researchers, clinicians, and patients. This perspective highlights seven key areas that are in need of further informatics research and innovation to support the realization of PM.

Despite years of preclinical development, biological interventions designed to treat complex diseases such as asthma often fail in phase III clinical trials. These failures suggest that current methods to analyze biomedical data might be missing critical aspects of biological complexity such as the assumption that cases and controls come from homogeneous distributions. Here we discuss why and how methods from the rapidly evolving field of visual analytics can help translational teams (consisting of biologists, clinicians, and bioinformaticians) to address the challenge of modeling and inferring heterogeneity in the proteomic and phenotypic profiles of patients with complex diseases. Because a primary goal of visual analytics is to amplify the cognitive capacities of humans for detecting patterns in complex data, we begin with an overview of the cognitive foundations for the field of visual analytics. Next, we organize the primary ways in which a specific form of visual analytics called networks has been used to model and infer biological mechanisms, which help to identify the properties of networks that are particularly useful for the discovery and analysis of proteomic heterogeneity in complex diseases. We describe one such approach called subject‐protein networks, and demonstrate its application on two proteomic datasets. This demonstration provides insights to help translational teams overcome theoretical, practical, and pedagogical hurdles for the widespread use of subject‐protein networks for analyzing molecular heterogeneities, with the translational goal of designing biomarker‐based clinical trials, and accelerating the development of personalized approaches to medicine.