Citations

Abstract Despite recent methodology advancements in clinical natural language processing (NLP), the adoption of clinical NLP models within the translational research community remains hindered by process heterogeneity and human factor variations. Concurrently, these factors also dramatically increase the difficulty in developing NLP models in multi-site settings, which is necessary for algorithm robustness and generalizability. Here, we reported on our experience developing an NLP solution for Coronavirus Disease 2019 (COVID-19) signs and symptom extraction in an open NLP framework from a subset of sites participating in the National COVID Cohort (N3C). We then empirically highlight the benefits of multi-site data for both symbolic and statistical methods, as well as highlight the need for federated annotation and evaluation to resolve several pitfalls encountered in the course of these efforts.

Abstract Introduction: Despite the decline in overall mortality and incidence of cancer in the US population, disparities in cancer care still largely exist within certain groups. The proportion of racial and ethnic minorities recruited to participate in cancer research is persistently lower than the US population. The rapid adoption of electronic health records (EHRs) systems has enabled computational cohort identification and eligibility screening for facilitating cancer research. However, EHR data is known to suffer from data quality issues due to numerous factors, such as care access. Patients with limited care access due to low socioeconomic status would naturally be sparsely represented in EHR data relative to those of higher status. Prospective enrollees may be disqualified due to biased or incomplete EHR data at the screening stage. Methods: To systematically identify data quality variabilities in EHRs, we implemented an information score (i-score) to reflect the density, variability, and irregularity of patients’ data. The pattern can be measured by estimating the variability of the time gaps between observations. For a patient observed n times, the encountered observation times are represented as x(1)…x(n). The relative time interval g for each observed encounter can be defined as g1 = [x(i+1)-x(i)]/[x(n)-x(1)], for I = 1…n - 1. The average amount of information of each observation is then defined as I, between 0 and 1, calculated as: I = 2/n+(n-2)/n[1-sqrt((n-1)Var{gi;i=1…n-1})]. An equality-spaced observation would receive a high i-score. The lower the i-score, the higher the data irregularity. We conducted a case study in a cohort of de novo stage IV breast cancer (DNIV) assembled by a previously validated phenotyping algorithm (Wang AACR 2020). The cohort contains 1,918 DNIV cases between 2004 and 2018 at Mayo Clinic Rochester. We used the i-score to assess patient visits, breast cancer diagnosis codes, and narrative documentation, achieved by the extraction of stage, surgery, de novo, and metastasis information based on the phenotyping definitions. The summary statistics of the i-scores were compared among four racial groups: African American (3%), Asian (1%), Caucasian (87%), and Other (9%). ANOVA F test was used to compare the group differences. Results and Conclusion: The mean i-scores for visit, diagnosis and narrative documentation were 0.712, 0.200, and 0.056 for African American, 0.412, 0.301, and 0.073 for Asian, 0.438, 0.278, and 0.056 for Caucasian, and 0.519, 0.343, and 0.043 for the other category, respectively. The F test indicated significant differences between visit (p<0.001) and diagnosis (p=0.0267) data. No difference was found in narrative documentation (p=0. 8591). In conclusion, there was a substantial i-score variation discovered, especially within the African American group, suggesting a strong need to address EHR quality disparity for cancer research. Citation Format: Sunyang Fu, Liwei Wang, Folakemi T. Odedina, Hongfang Liu. Assessment of EHR data quality variabilities among different racial groups in the cohort of de novo stage IV breast cancer [abstract]. In: Proceedings of the 15th AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2022 Sep 16-19; Philadelphia, PA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2022;31(1 Suppl):Abstract nr A020.

AbstractAn increasing number of studies have reported using natural language processing (NLP) to assist observational research by extracting clinical information from electronic health records (EHRs). Currently, no standardized reporting guidelines for NLP‐assisted observational studies exist. The absence of detailed reporting guidelines may create ambiguity in the use of NLP‐derived content, knowledge gaps in the current research reporting practices, and reproducibility challenges. To address these issues, we conducted a scoping review of NLP‐assisted observational clinical studies and examined their reporting practices, focusing on NLP methodology and evaluation. Through our investigation, we discovered a high variation regarding the reporting practices, such as inconsistent use of references for measurement studies, variation in the reporting location (reference, appendix, and manuscript), and different granularity of NLP methodology and evaluation details. To promote the wide adoption and utilization of NLP solutions in clinical research, we outline several perspectives that align with the six principles released by the World Health Organization (WHO) that guide the ethical use of artificial intelligence for health.

PURPOSEThe advancement of natural language processing (NLP) has promoted the use of detailed textual data in electronic health records (EHRs) to support cancer research and to facilitate patient care. In this review, we aim to assess EHR for cancer research and patient care by using the Minimal Common Oncology Data Elements (mCODE), which is a community-driven effort to define a minimal set of data elements for cancer research and practice. Specifically, we aim to assess the alignment of NLP-extracted data elements with mCODE and review existing NLP methodologies for extracting said data elements.METHODSPublished literature studies were searched to retrieve cancer-related NLP articles that were written in English and published between January 2010 and September 2020 from main literature databases. After the retrieval, articles with EHRs as the data source were manually identified. A charting form was developed for relevant study analysis and used to categorize data including four main topics: metadata, EHR data and targeted cancer types, NLP methodology, and oncology data elements and standards.RESULTSA total of 123 publications were selected finally and included in our analysis. We found that cancer research and patient care require some data elements beyond mCODE as expected. Transparency and reproductivity are not sufficient in NLP methods, and inconsistency in NLP evaluation exists.CONCLUSIONWe conducted a comprehensive review of cancer NLP for research and patient care using EHRs data. Issues and barriers for wide adoption of cancer NLP were identified and discussed.

Abstract Objective Accurate extraction of breast cancer patients’ phenotypes is important for clinical decision support and clinical research. This study developed and evaluated cancer domain pretrained CancerBERT models for extracting breast cancer phenotypes from clinical texts. We also investigated the effect of customized cancer-related vocabulary on the performance of CancerBERT models. Materials and Methods A cancer-related corpus of breast cancer patients was extracted from the electronic health records of a local hospital. We annotated named entities in 200 pathology reports and 50 clinical notes for 8 cancer phenotypes for fine-tuning and evaluation. We kept pretraining the BlueBERT model on the cancer corpus with expanded vocabularies (using both term frequency-based and manually reviewed methods) to obtain CancerBERT models. The CancerBERT models were evaluated and compared with other baseline models on the cancer phenotype extraction task. Results All CancerBERT models outperformed all other models on the cancer phenotyping NER task. Both CancerBERT models with customized vocabularies outperformed the CancerBERT with the original BERT vocabulary. The CancerBERT model with manually reviewed customized vocabulary achieved the best performance with macro F1 scores equal to 0.876 (95% CI, 0.873–0.879) and 0.904 (95% CI, 0.902–0.906) for exact match and lenient match, respectively. Conclusions The CancerBERT models were developed to extract the cancer phenotypes in clinical notes and pathology reports. The results validated that using customized vocabulary may further improve the performances of domain specific BERT models in clinical NLP tasks. The CancerBERT models developed in the study would further help clinical decision support.

AbstractPurposeRural populations are disproportionately affected by the COVID‐19 pandemic. We characterized urban‐rural disparities in patient portal messaging utilization for COVID‐19, and, of those who used the portal during its early stage in the Midwest.MethodsWe collected over 1 million portal messages generated by midwestern Mayo Clinic patients from February to August 2020. We analyzed patient‐generated messages (PGMs) on COVID‐19 by urban‐rural locality and incorporated patients’ sociodemographic factors into the analysis.FindingsThe urban‐rural ratio of portal users, message senders, and COVID‐19 message senders was 1.18, 1.31, and 1.79, indicating greater use among urban patients. The urban‐rural ratio (1.69) of PGMs on COVID‐19 was higher than that (1.43) of general PGMs. The urban‐rural ratios of messaging were 1.72‐1.85 for COVID‐19‐related care and 1.43‐1.66 for other health care issues on COVID‐19. Compared with urban patients, rural patients sent fewer messages for COVID‐19 diagnosis and treatment but more messages for other reasons related to COVID‐19‐related health care (eg, isolation and anxiety). The frequent senders of COVID‐19‐related messages among rural patients were 40+ years old, women, married, and White.ConclusionsIn this Midwest health system, rural patients were less likely to use patient online services during a pandemic and their reasons for its use differ from urban patients. Results suggest opportunities for increasing equity in rural patient engagement in patient portals (in particular, minority populations) for COVID‐19. Public health intervention strategies could target reasons why rural patients might seek health care in a pandemic, such as social isolation and anxiety.

AbstractSummaryBuilding a high-quality annotation corpus requires expenditure of considerable time and expertise, particularly for biomedical and clinical research applications. Most existing annotation tools provide many advanced features to cover a variety of needs where the installation, integration and difficulty of use present a significant burden for actual annotation tasks. Here, we present MedTator, a serverless annotation tool, aiming to provide an intuitive and interactive user interface that focuses on the core steps related to corpus annotation, such as document annotation, corpus summarization, annotation export and annotation adjudication.Availability and implementationMedTator and its tutorial are freely available from https://ohnlp.github.io/MedTator. MedTator source code is available under the Apache 2.0 license: https://github.com/OHNLP/MedTator.Supplementary informationSupplementary data are available at Bioinformatics online.

Introduction: Proton pump inhibitors (PPIs) are used by an estimated 29 million Americans. PPIs increase the levels of asymmetrical dimethylarginine, a known risk factor for cardiovascular disease (CVD). Data from a select population of patients with CVD suggest that PPI use is associated with an increased risk of stroke, heart failure, and coronary heart disease. The impact of PPI use on incident CVD is largely unknown in the general population. Hypothesis: We hypothesized that PPI users have a higher risk of incident total CVD, coronary heart disease, stroke, and heart failure compared to nonusers. To demonstrate specificity of association, we additionally hypothesized that there is not an association between use of H 2 -blockers - another commonly used class of medications with similar indications as PPIs - and CVD. Methods: We used the Rochester Epidemiology Project’s medical records-linkage system to identify all residents of Olmsted County, MN on our baseline date of January 1, 2004 (N=140217). We excluded persons who did not grant permission for their records to be used for research, were <18 years old, had a history of CVD, had missing data for any variable included in our model, or had evidence of PPI use within the previous year.We followed our final cohort (N=58175) for up to 12 years. The administrative censoring date for CVD was 1/20/2014, for coronary heart disease was 8/3/2016, for stroke was 9/9/2016, and for heart failure was 1/20/2014. Time-varying PPI ever-use was ascertained using 1) natural language processing to capture unstructured text from the electronic health record, and 2) outpatient prescriptions. An incident CVD event was defined as the first occurrence of 1) validated heart failure, 2) validated coronary heart disease, or 3) stroke, defined using diagnostic codes only. As a secondary analysis, we calculated the association between time-varying H 2 -blocker ever-use and CVD among persons not using H 2 -blockers at baseline. Results: After adjustment for age, sex, race, education, hypertension, hyperlipidemia, diabetes, and body-mass-index, PPI use was associated with an approximately 50% higher risk of CVD (hazard ratio [95% CI]: 1.51 [1.37-1.67]; 2187 CVD events), stroke (hazard ratio [95% CI]: 1.49 [1.35-1.65]; 1928 stroke events), and heart failure (hazard ratio [95% CI]: 1.56 [1.23-1.97]; 353 heart failure events) compared to nonusers. Users of PPIs had a 35% greater risk of coronary heart disease than nonusers (95% CI: 1.13-1.61; 626 coronary heart disease events). Use of H 2 -blockers was also associated with a higher risk of CVD (adjusted hazard ratio [95% CI]: 1.23 [1.08-1.41]; 2331 CVD events). Conclusions: PPI use is associated with a higher risk of CVD, coronary heart disease, stroke and heart failure. Use of a drug with no known cardiac toxicity - H 2 -blockers - was also associated with a greater risk of CVD, warranting further study.