Zong, Nansu

Abstract Background Alzheimer’s disease (AD) is a progressive neurological disorder with no specific curative medications. Sophisticated clinical skills are crucial to optimize treatment regimens given the multiple coexisting comorbidities in the patient population. Objective Here, we propose a study to leverage reinforcement learning (RL) to learn the clinicians’ decisions for AD patients based on the longitude data from electronic health records. Methods In this study, we selected 1736 patients from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. We focused on the two most frequent concomitant diseases—depression, and hypertension, thus creating 5 data cohorts (ie, Whole Data, AD, AD-Hypertension, AD-Depression, and AD-Depression-Hypertension). We modeled the treatment learning into an RL problem by defining states, actions, and rewards. We built a regression model and decision tree to generate multiple states, used six combinations of medications (ie, cholinesterase inhibitors, memantine, memantine-cholinesterase inhibitors, hypertension drugs, supplements, or no drugs) as actions, and Mini-Mental State Exam (MMSE) scores as rewards. Results Given the proper dataset, the RL model can generate an optimal policy (regimen plan) that outperforms the clinician’s treatment regimen. Optimal policies (ie, policy iteration and Q-learning) had lower rewards than the clinician’s policy (mean −3.03 and −2.93 vs. −2.93, respectively) for smaller datasets but had higher rewards for larger datasets (mean −4.68 and −2.82 vs. −4.57, respectively). Conclusions Our results highlight the potential of using RL to generate the optimal treatment based on the patients’ longitude records. Our work can lead the path towards developing RL-based decision support systems that could help manage AD with comorbidities.

e13552 Background: Real-world data from Electronic Health Records (EHR) have been widely used for patient identification to build study cohorts for clinical research. Traditionally, diagnosis codes in the EHR, such International Classification of Diseases (ICD), are used to identify the target patients. However, the accuracy of this approach is dependent on the accuracy of ICD coding, with potential errors especially for tumor types that are frequent locations for metastases (which may contribute to mis-coding). In this study, we attempted to develop a Machine Learning (ML) based approach on EHR data to improve the accuracy of identification of patients with lung cancer. Methods: We used survey respondents in the Enhanced, EHR-facilitated Cancer Symptom Control (E2C2, NCT03892967) cluster-randomized trial at Mayo Clinic as our initial pan-cancer cohort. E2C2 includes adults receiving Medical Hematology/Oncology care for a solid or liquid tumor at Mayo Clinic. We collected cancer diagnoses from the individually abstracted Mayo Clinic Cancer Registry to annotate the cancer type for the patients. Lung cancer related ICD-9 (162.X) and ICD-10 (C34.X) codes were used to build a search query on the Mayo’s EHR to find target patients from the E2C2 cohort, and to investigate the ICD-based lung cancer patient identification performance. Diagnosis, radiation oncology treatment (CPT 77261 - 77799), and antineoplastic drug administration data were collected from EHR as variables. Logistic Regression (LR), Support vector machine (SVM), Random Forest (RF), and eXtreme Gradient Boosting (XGB) were selected to build models for lung cancer patient identification. 10-fold cross-validation was implemented to assess the models. Precision, Recall, F1 Score, and area under the curve (AUC) were selected to measure the performance. Results: We collected 13,893 patients with a specific cancer diagnosis, and 1,394 were identified as having lung cancer. The identification performance across different methods is shown. The ICD-based method only had 0.65 precision. It means we collected a lot of false positive cases (other cancer but no lung cancer patient), as we mentioned in the background. SVM gets the best precision results, but its recall and F1 score are not good enough. XGB shows the best F1 Score and AUC, which also means this method achieved the best and most balanced performance. Conclusions: In this study, we found that XGB-based methods achieved the best identification performance for lung cancer. In the future, we will investigate if this is also true for the identification of other cancer types. [Table: see text]

AbstractAn increasing number of studies have reported using natural language processing (NLP) to assist observational research by extracting clinical information from electronic health records (EHRs). Currently, no standardized reporting guidelines for NLP‐assisted observational studies exist. The absence of detailed reporting guidelines may create ambiguity in the use of NLP‐derived content, knowledge gaps in the current research reporting practices, and reproducibility challenges. To address these issues, we conducted a scoping review of NLP‐assisted observational clinical studies and examined their reporting practices, focusing on NLP methodology and evaluation. Through our investigation, we discovered a high variation regarding the reporting practices, such as inconsistent use of references for measurement studies, variation in the reporting location (reference, appendix, and manuscript), and different granularity of NLP methodology and evaluation details. To promote the wide adoption and utilization of NLP solutions in clinical research, we outline several perspectives that align with the six principles released by the World Health Organization (WHO) that guide the ethical use of artificial intelligence for health.

Reproducibility is an important quality criterion for the secondary use of electronic health records (EHRs). However, multiple barriers to reproducibility are embedded in the heterogeneous EHR environment. These barriers include complex processes for collecting and organizing EHR data and dynamic multi-level interactions occurring during information use (e.g., inter-personal, inter-system, and cross-institutional). To ensure reproducible use of EHRs, we investigated four information quality dimensions and examine the implications for reproducibility based on a real-world EHR study. Four types of IQ measurements suggested that barriers to reproducibility occurred for all stages of secondary use of EHR data. We discussed our recommendations and emphasized the importance of promoting transparent, high-throughput, and accessible data infrastructures and implementation best practices (e.g., data quality assessment, reporting standard).

Abstract Internal validation is the most popular evaluation strategy used for drug–target predictive models. The simple random shuffling in the cross-validation, however, is not always ideal to handle large, diverse and copious datasets as it could potentially introduce bias. Hence, these predictive models cannot be comprehensively evaluated to provide insight into their general performance on a variety of use-cases (e.g. permutations of different levels of connectiveness and categories in drug and target space, as well as validations based on different data sources). In this work, we introduce a benchmark, BETA, that aims to address this gap by (i) providing an extensive multipartite network consisting of 0.97 million biomedical concepts and 8.5 million associations, in addition to 62 million drug–drug and protein–protein similarities and (ii) presenting evaluation strategies that reflect seven cases (i.e. general, screening with different connectivity, target and drug screening based on categories, searching for specific drugs and targets and drug repurposing for specific diseases), a total of seven Tests (consisting of 344 Tasks in total) across multiple sampling and validation strategies. Six state-of-the-art methods covering two broad input data types (chemical structure- and gene sequence-based and network-based) were tested across all the developed Tasks. The best-worst performing cases have been analyzed to demonstrate the ability of the proposed benchmark to identify limitations of the tested methods for running over the benchmark tasks. The results highlight BETA as a benchmark in the selection of computational strategies for drug repurposing and target discovery.