Citations

Abstract Despite recent methodology advancements in clinical natural language processing (NLP), the adoption of clinical NLP models within the translational research community remains hindered by process heterogeneity and human factor variations. Concurrently, these factors also dramatically increase the difficulty in developing NLP models in multi-site settings, which is necessary for algorithm robustness and generalizability. Here, we reported on our experience developing an NLP solution for Coronavirus Disease 2019 (COVID-19) signs and symptom extraction in an open NLP framework from a subset of sites participating in the National COVID Cohort (N3C). We then empirically highlight the benefits of multi-site data for both symbolic and statistical methods, as well as highlight the need for federated annotation and evaluation to resolve several pitfalls encountered in the course of these efforts.

Abstract Objective Social determinants of health (SDoH) play critical roles in health outcomes and well-being. Understanding the interplay of SDoH and health outcomes is critical to reducing healthcare inequalities and transforming a “sick care” system into a “health-promoting” system. To address the SDOH terminology gap and better embed relevant elements in advanced biomedical informatics, we propose an SDoH ontology (SDoHO), which represents fundamental SDoH factors and their relationships in a standardized and measurable way. Material and Methods Drawing on the content of existing ontologies relevant to certain aspects of SDoH, we used a top-down approach to formally model classes, relationships, and constraints based on multiple SDoH-related resources. Expert review and coverage evaluation, using a bottom-up approach employing clinical notes data and a national survey, were performed. Results We constructed the SDoHO with 708 classes, 106 object properties, and 20 data properties, with 1,561 logical axioms and 976 declaration axioms in the current version. Three experts achieved 0.967 agreement in the semantic evaluation of the ontology. A comparison between the coverage of the ontology and SDOH concepts in 2 sets of clinical notes and a national survey instrument also showed satisfactory results. Discussion SDoHO could potentially play an essential role in providing a foundation for a comprehensive understanding of the associations between SDoH and health outcomes and paving the way for health equity across populations. Conclusion SDoHO has well-designed hierarchies, practical objective properties, and versatile functionalities, and the comprehensive semantic and coverage evaluation achieved promising performance compared to the existing ontologies relevant to SDoH.

Abstract Background: Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. Objectives: The main objective of this review is to identify and analyze published research on clinical IR, including the methods, techniques, and tools used to retrieve and analyze clinical information from various sources. We aim to provide a comprehensive overview of the current state of clinical IR research and guide future research efforts in this field. Methods: We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and conducted a thorough search of multiple databases, including Ovid Embase, Ovid Cochrane Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, from January 1st, 2012, to January 4th, 2023. The screening process involved multiple reviewers, and we included 184 papers for the final review. Results: We conducted a detailed analysis and discussion of various aspects of clinical IR research, including publication year, data sources, methods, techniques, evaluation metrics, shared tasks, and applications. Our analysis revealed key research areas in clinical IR, such as indexing, ranking, and query expansion, and identified opportunities for future research in these areas.

Multiplex immunofluorescence (MxIF) images provide detailed information of cell composition and spatial context for biomedical research. However, compromised data quality could lead to research biases. Comprehensive image quality checking (QC) is essential for reliable downstream analysis. As a reliable and specific staining of cell nuclei, 4′,6-diamidino-2-phenylindole (DAPI) signals were used as references for tissue localization and auto-focusing across MxIF staining–scanning–bleaching iterations and could potentially be reused for QC. To confirm the feasibility of using DAPI as QC reference, pixel-level DAPI values were extracted to calculate signal fluctuations and tissue content similarities in staining–scanning–bleaching iterations for identifying quality issues. Concordance between automatic quantification and human experts’ annotations were evaluated on a data set consisting of 348 fields of view (FOVs) with 45 immune and tumor cell markers. Cell distribution differences between subsets of QC-pass vs QC-failed FOVs were compared to investigate the downstream effects. Results showed that 87.3% FOVs with tissue damage and 73.4% of artifacts were identified. QC-failed FOVs showed elevated regional gathering in cellular feature space compared with the QC-pass FOVs. Our results supported that DAPI signals could be used as references for MxIF image QC, and low-quality FOVs identified by our method must be cautiously considered for downstream analyses.

Importance The effectiveness of triplet therapy compared with androgen pathway inhibitor (API) doublets in a heterogeneous patient population with metastatic castration-sensitive prostate cancer (mCSPC) is unknown. Objective To assess the comparative effectiveness of contemporary systemic treatment options for patients with mCSPC across clinically relevant subgroups. Data Sources For this systematic review and meta-analysis, Ovid MEDLINE and Embase were searched from each database’s inception (MEDLINE, 1946; Embase, 1974) through June 16, 2021. Subsequently, a “living” auto search was created with weekly updates to identify new evidence as it became available. Study Selection Phase 3 randomized clinical trials (RCTs) assessing first-line treatment options for mCSPC. Data Extraction and Synthesis Two independent reviewers extracted data from eligible RCTs. The comparative effectiveness of different treatment options was assessed with a fixed-effect network meta-analysis. Data were analyzed on July 10, 2022. Main Outcomes and Measures Outcomes of interest included overall survival (OS), progression-free survival (PFS), grade 3 or higher adverse events, and health-related quality of life. Results This report included 10 RCTs with 11 043 patients and 9 unique treatment groups. Median ages of the included population ranged from 63 to 70 years. Current evidence for the overall population suggests that the darolutamide (DARO) triplet (DARO + docetaxel [D] + androgen deprivation therapy [ADT]; hazard ratio [HR], 0.68; 95% CI, 0.57-0.81), as well as the abiraterone (AAP) triplet (AAP + D + ADT; HR, 0.75; 95% CI, 0.59-0.95), are associated with improved OS compared with D doublet (D + ADT) but not compared with API doublets. Among patients with high-volume disease, AAP + D + ADT may improve OS compared with D + ADT (HR, 0.72; 95% CI, 0.55-0.95) but not compared with AAP + ADT, enzalutamide (E) + ADT, and apalutamide (APA) + ADT. For patients with low-volume disease, AAP + D + ADT may not improve OS compared with APA + ADT, AAP + ADT, E + ADT, and D + ADT. Conclusions and Relevance The potential benefit observed with triplet therapy must be interpreted with careful accounting for the volume of disease and the choice of doublet comparisons used in the clinical trials. These findings suggest an equipoise to how triplet regimens compare with API doublet combinations and provide direction for future clinical trials.

Abstract Introduction: Despite the decline in overall mortality and incidence of cancer in the US population, disparities in cancer care still largely exist within certain groups. The proportion of racial and ethnic minorities recruited to participate in cancer research is persistently lower than the US population. The rapid adoption of electronic health records (EHRs) systems has enabled computational cohort identification and eligibility screening for facilitating cancer research. However, EHR data is known to suffer from data quality issues due to numerous factors, such as care access. Patients with limited care access due to low socioeconomic status would naturally be sparsely represented in EHR data relative to those of higher status. Prospective enrollees may be disqualified due to biased or incomplete EHR data at the screening stage. Methods: To systematically identify data quality variabilities in EHRs, we implemented an information score (i-score) to reflect the density, variability, and irregularity of patients’ data. The pattern can be measured by estimating the variability of the time gaps between observations. For a patient observed n times, the encountered observation times are represented as x(1)…x(n). The relative time interval g for each observed encounter can be defined as g1 = [x(i+1)-x(i)]/[x(n)-x(1)], for I = 1…n - 1. The average amount of information of each observation is then defined as I, between 0 and 1, calculated as: I = 2/n+(n-2)/n[1-sqrt((n-1)Var{gi;i=1…n-1})]. An equality-spaced observation would receive a high i-score. The lower the i-score, the higher the data irregularity. We conducted a case study in a cohort of de novo stage IV breast cancer (DNIV) assembled by a previously validated phenotyping algorithm (Wang AACR 2020). The cohort contains 1,918 DNIV cases between 2004 and 2018 at Mayo Clinic Rochester. We used the i-score to assess patient visits, breast cancer diagnosis codes, and narrative documentation, achieved by the extraction of stage, surgery, de novo, and metastasis information based on the phenotyping definitions. The summary statistics of the i-scores were compared among four racial groups: African American (3%), Asian (1%), Caucasian (87%), and Other (9%). ANOVA F test was used to compare the group differences. Results and Conclusion: The mean i-scores for visit, diagnosis and narrative documentation were 0.712, 0.200, and 0.056 for African American, 0.412, 0.301, and 0.073 for Asian, 0.438, 0.278, and 0.056 for Caucasian, and 0.519, 0.343, and 0.043 for the other category, respectively. The F test indicated significant differences between visit (p<0.001) and diagnosis (p=0.0267) data. No difference was found in narrative documentation (p=0. 8591). In conclusion, there was a substantial i-score variation discovered, especially within the African American group, suggesting a strong need to address EHR quality disparity for cancer research. Citation Format: Sunyang Fu, Liwei Wang, Folakemi T. Odedina, Hongfang Liu. Assessment of EHR data quality variabilities among different racial groups in the cohort of de novo stage IV breast cancer [abstract]. In: Proceedings of the 15th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2022 Sep 16-19; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2022;31(1 Suppl):Abstract nr A020.

Abstract Context Metformin is the first-line drug for treating diabetes but has a high failure rate. Objective To identify demographic and clinical factors available in the electronic health record (EHR) that predict metformin failure. Methods A cohort of patients with at least 1 abnormal diabetes screening test that initiated metformin was identified at 3 sites (Arizona, Mississippi, and Minnesota). We identified 22 047 metformin initiators (48% female, mean age of 57 ± 14 years) including 2141 African Americans, 440 Asians, 962 Other/Multiracial, 1539 Hispanics, and 16 764 non-Hispanic White people. We defined metformin failure as either the lack of a target glycated hemoglobin (HbA1c) (<7%) within 18 months of index or the start of dual therapy. We used tree-based extreme gradient boosting (XGBoost) models to assess overall risk prediction performance and relative contribution of individual factors when using EHR data for risk of metformin failure. Results In this large diverse population, we observed a high rate of metformin failure (43%). The XGBoost model that included baseline HbA1c, age, sex, and race/ethnicity corresponded to high discrimination performance (C-index of 0.731; 95% CI 0.722, 0.740) for risk of metformin failure. Baseline HbA1c corresponded to the largest feature performance with higher levels associated with metformin failure. The addition of other clinical factors improved model performance (0.745; 95% CI 0.737, 0.754, P < .0001). Conclusion Baseline HbA1c was the strongest predictor of metformin failure and additional factors substantially improved performance suggesting that routinely available clinical data could be used to identify patients at high risk of metformin failure who might benefit from closer monitoring and earlier treatment intensification.

Background Covert cerebrovascular disease (CCD) has been shown to be associated with dementia in population‐based studies with magnetic resonance imaging (MRI) screening, but dementia risk associated with incidentally discovered CCD is not known. Methods and Results Individuals aged ≥50 years enrolled in the Kaiser Permanente Southern California health system receiving head computed tomography (CT) or MRI for nonstroke indications from 2009 to 2019, without prior ischemic stroke/transient ischemic attack, dementia/Alzheimer disease, or visit reason/scan indication suggestive of cognitive decline or stroke were included. Natural language processing identified incidentally discovered covert brain infarction (id‐CBI) and white matter disease (id‐WMD) on the neuroimage report; white matter disease was characterized as mild, moderate, severe, or undetermined. We estimated risk of dementia associated with id‐CBI and id‐WMD. Among 241 050 qualified individuals, natural language processing identified 69 931 (29.0%) with id‐WMD and 11 328 (4.7%) with id‐CBI. Dementia incidence rates (per 1000 person‐years) were 23.5 (95% CI, 22.9–24.0) for patients with id‐WMD, 29.4 (95% CI, 27.9–31.0) with id‐CBI, and 6.0 (95% CI, 5.8–6.2) without id‐CCD. The association of id‐WMD with future dementia was stronger in younger (aged <70 years) versus older (aged ≥70 years) patients and for CT‐ versus MRI‐discovered lesions. For patients with versus without id‐WMD on CT, the adjusted HR was 2.87 (95% CI, 2.58–3.19) for younger and 1.87 (95% CI, 1.79–1.95) for older patients. For patients with versus without id‐WMD on MRI, the adjusted HR for dementia risk was 2.28 (95% CI, 1.99–2.62) for younger and 1.48 (95% CI, 1.32–1.66) for older patients. The adjusted HR for id‐CBI was 2.02 (95% CI, 1.70–2.41) for younger and 1.22 (95% CI, 1.15–1.30) for older patients for either modality. Dementia risk was strongly correlated with id‐WMD severity; adjusted HRs compared with patients who were negative for id‐WMD by MRI ranged from 1.41 (95% CI, 1.25–1.60) for those with mild disease on MRI to 4.11 (95% CI, 3.58–4.72) for those with severe disease on CT. Conclusions Incidentally discovered CCD is common and associated with a high risk of dementia, representing an opportunity for prevention. The association is strengthened when discovered at younger age, by increasing id‐WMD severity, and when id‐WMD is detected by CT scan rather than MRI.

AbstractAn increasing number of studies have reported using natural language processing (NLP) to assist observational research by extracting clinical information from electronic health records (EHRs). Currently, no standardized reporting guidelines for NLP‐assisted observational studies exist. The absence of detailed reporting guidelines may create ambiguity in the use of NLP‐derived content, knowledge gaps in the current research reporting practices, and reproducibility challenges. To address these issues, we conducted a scoping review of NLP‐assisted observational clinical studies and examined their reporting practices, focusing on NLP methodology and evaluation. Through our investigation, we discovered a high variation regarding the reporting practices, such as inconsistent use of references for measurement studies, variation in the reporting location (reference, appendix, and manuscript), and different granularity of NLP methodology and evaluation details. To promote the wide adoption and utilization of NLP solutions in clinical research, we outline several perspectives that align with the six principles released by the World Health Organization (WHO) that guide the ethical use of artificial intelligence for health.