Ruan, Xiaoyang

Abstract Error analysis is a critical step in evaluating and improving clinical concept extraction models, the most common clinical natural language processing (NLP) task. Unlike corpus annotation, which follows standardized protocols for creating gold-standard datasets, error analysis requires nuanced judgment grounded in both clinical expertise and NLP knowledge. This is especially important given the heterogeneity of clinical text, where variations in documentation style, note structure, and terminology can substantially influence model behavior. Despite its importance, there is currently no standardized, user-level framework to support systematic error analysis in clinical concept extraction task. In this study, we developed and validated MedError, a machine-assisted, human-in-the-loop framework designed to standardize and enhance error analysis for clinical concept extraction tasks. We collected and manually curated a corpus of 1,187 unique errors from a total of 4,237 notes across three different distinct hospitals. The error categories were defined using our previously validated error taxonomy and included 480 false negatives and 707 false positives across 25 error types and 48 clinical concept categories. We evaluated the performance of three proprietary and three open-source large language models (LLMs) in automatically classifying these errors into 26 and 15 predefined categories. We further developed a machine-assisted framework, MedError, which integrates best practices in error analysis, LLM-assisted classification and reasoning, and a user-friendly interface to enable more efficient, reproducible, and context-aware error analysis. The framework supports both single-site and federated multisite error analysis, facilitating the effective translation of clinical NLP systems into real-world settings.

PURPOSEThe advancement of natural language processing (NLP) has promoted the use of detailed textual data in electronic health records (EHRs) to support cancer research and to facilitate patient care. In this review, we aim to assess EHR for cancer research and patient care by using the Minimal Common Oncology Data Elements (mCODE), which is a community-driven effort to define a minimal set of data elements for cancer research and practice. Specifically, we aim to assess the alignment of NLP-extracted data elements with mCODE and review existing NLP methodologies for extracting said data elements.METHODSPublished literature studies were searched to retrieve cancer-related NLP articles that were written in English and published between January 2010 and September 2020 from main literature databases. After the retrieval, articles with EHRs as the data source were manually identified. A charting form was developed for relevant study analysis and used to categorize data including four main topics: metadata, EHR data and targeted cancer types, NLP methodology, and oncology data elements and standards.RESULTSA total of 123 publications were selected finally and included in our analysis. We found that cancer research and patient care require some data elements beyond mCODE as expected. Transparency and reproductivity are not sufficient in NLP methods, and inconsistency in NLP evaluation exists.CONCLUSIONWe conducted a comprehensive review of cancer NLP for research and patient care using EHRs data. Issues and barriers for wide adoption of cancer NLP were identified and discussed.

Objective The primary aim was to compare independent and joint performance of retrieving smoking status through different sources, including narrative text processed by natural language processing (NLP), patient-provided information (PPI), and diagnosis codes (ie, International Classification of Diseases, Ninth Revision [ICD-9]). We also compared the performance of retrieving smoking strength information (ie, heavy/light smoker) from narrative text and PPL Materials and Methods Our study leveraged an existing lung cancer cohort for smoking status, amount, and strength information, which was manually chart-reviewed. On the NLP side, smoking-related electronic medical record (EMR) data were retrieved first. A pattern-based smoking information extraction module was then implemented to extract smoking-related information. After that, heuristic rules were used to obtain smoking status-related information. Smoking information was also obtained from structured data sources based on diagnosis codes and PPI. Sensitivity, specificity, and accuracy were measured using patients with coverage (ie, the proportion of patients whose smoking status/strength can be effectively determined). Results NLP alone has the best overall performance for smoking status extraction (patient coverage: 0.88; sensitivity: 0.97; specificity: 0.70; accuracy: 0.88); combining PPI with NLP further improved patient coverage to 0.96. ICD-9 does not provide additional improvement to NLP and its combination with PPI. For smoking strength, combining NLP with PPI has slight improvement over NLP alone. Conclusion These findings suggest that narrative text could serve as a more reliable and comprehensive source for obtaining smoking-related information than structured data sources. PPI, the readily available structured data, could be used as a complementary source for more comprehensive patient coverage.

Chromosome instability (CIN) is widely observed in both sporadic and hereditary colorectal cancer (CRC). Defects in APC and WNT signaling are primarily associated with CIN in hereditary CRC, but the genetic causes for CIN in sporadic CRC remain elusive. Using high‐density SNP array and exome data from The Cancer Genome Atlas (TCGA), we characterized loss of heterozygosity (LOH) and copy number variation (CNV) in the peripheral blood, normal colon, and corresponding tumor tissue in 15 CRC patients with proficient mismatch repair (MMR) and 24 CRC patients with deficient MMR. We found a high frequency of 18q LOH in tumors and arm‐specific enrichment of genetic aberrations on 18q in the normal colon (primarily copy neutral LOH) and blood (primarily copy gain). These aberrations were specific to the sporadic, pMMR CRC. Though in tumor samples genetic aberrations were observed for genes commonly mutated in hereditary CRC (eg, APC, CTNNB1, SMAD4, BRAF), none of them showed LOH or CNV in the normal colon or blood. DCC located on 18q21.1 topped the list of genes with genetic aberrations in the tumor. In an independent cohort of 13 patients subjected to Whole Genome Sequencing (WGS), we found LOH and CNV on 18q in adenomatous polyp and tumor tissues. Our data suggests that patients with sporadic CRC may have genetic aberrations preferentially enriched on 18q in their blood, normal colon epithelium, and non‐malignant polyp lesions that may prove useful as a clinical marker for sporadic CRC detection and risk assessment.

Abstract Despite advances in prevention and early detection strategies, colorectal cancer (CRC) remains the second leading cause of cancer death in the United States. The majority of CRC arises from adenomatous polyps. However, only 5% of all people who receive a colonoscopy will have a lesion determined to be at high risk for malignant transformation. Adenomas are primarily classified based on histology, size, degree of dysplasia and limited genetic mutation information. Further characterization of what contributes to or underlies the malignant transformation of polyps is currently lacking, but is essential to further understand the transition from polyp to cancer and factors that lead only a small portion of polyps to progress onto cancer. Although telomere length is a recognized biomarker in multiple cancers, few studies have addressed the comprehensive role of telomeres in CRC progression. Telomere shortening is a fundamental feature of dividing cells related to the age of the cell and is implicated in carcinogenesis. Telomere length is not solely determined by attrition resulting from cell division, but also through telomere maintenance mechanisms (TMM), which are often exploited in malignant cells. In this study, we will measure the telomere profile (consisting of telomere length and TMM) in normal colon epithelium, polyps and the adjacent tumor trios and normal colon epithelium and cancer-free polyp pairs. Our aim is to determine which telomere features distinguish cancer adjacent polyps from cancer free polyps. This is an innovative approach for identifying the definable telomere-related properties that distinguish polyps with malignant potential that remain benign from those that transform to cancer and progress to invasive lesions. Further characterization of the molecular profile of polyps undergoing the transition to cancer that differentiate cancer free polyps has the potential to be utilized in designing optimal treatment options for patients with polyps that carry the risk of CRC. Citation Format: Brooke R. Druliner, Ruth Johnson, Xiaoyang Ruan, Russell Vanderboom, Donna Felmlee Devine, Jill Washechek Aletto, Lisa A. Boardman. Time lapse to cancer: Defining the transition from polyp to colorectal cancer based on telomere metrics. [abstract]. In: Proceedings of the AACR Special Conference on Computational and Systems Biology of Cancer; Feb 8-11 2015; San Francisco, CA. Philadelphia (PA): AACR; Cancer Res 2015;75(22 Suppl 2):Abstract nr B2-04.

Abstract Introduction: Activation of telomerase and telomere lengthening in patient peripheral blood leukocytes (PBL) is associated with longer survival in cancer patients. There are few studies to date that have measured telomerase activity in clinical PBL samples, and it is unclear how initial collection and processing of PBL samples impact the measurement of telomerase activity. In this study, we assessed if telomerase activity measurement varies based on the method of blood collection, processing and storage. Methods: We first collected whole blood drawn in EDTA tubes from two healthy patients that were then processed three different ways. The first whole blood sample was spun down and the buffy coat was pipetted from the top of the red blood cells, and then placed directly into the -80°C freezer. The next two whole blood samples were isolated using an ACCUSPIN™ tube, which utilizes density gradient separation to recover the PBL without contamination of red blood cells. The resulting PBL from these two samples were suspended in freezing media containing RPMI and DMSO, with one sample also containing penicillin streptomycin glutamine, and then both samples were stored in liquid nitrogen following a slow freeze. We then measured enzymatic telomerase activity in each of the three PBL samples from the two different patients. Results: We detected a comparable level of telomerase activity in all three PBL samples. These findings indicate that telomerase activity can be measured in buffy coat samples from patients independent of how the samples were processed and stored. Conclusions: These findings are important because many biorepositories may not have collected or processed PBL samples by identical processes, therefore not allowing direct comparison across samples. Additionally, our finding that telomerase activity levels are comparable between PBL samples containing red blood cells and those processed using an ACCUSPIN™ tube is valuable because it eliminates the necessity for the extra cost of the ACCUSPIN™ tube, freezing media, antibiotics and use of liquid nitrogen storage tanks. Future studies will focus on the determining the relationship of telomerase activity between the PBL, normal colon and cancer tissue in colorectal cancer patients. Citation Format: Ruth A. Johnson, Brooke R. Druliner, Jill Washechek Alleto, Donna Felmlee Devine, Xiaoyang Ruan, Russell Vanderboom, Lisa A. Boardman. How collection, processing and storage of PBL samples impact the measurement of telomerase activity. [abstract]. In: Proceedings of the AACR Special Conference on Computational and Systems Biology of Cancer; Feb 8-11 2015; San Francisco, CA. Philadelphia (PA): AACR; Cancer Res 2015;75(22 Suppl 2):Abstract nr B1-46.

Abstract Objectives Drug repurposing, which finds new indications for existing drugs, has received great attention recently. The goal of our work is to assess the feasibility of using electronic health records (EHRs) and automated informatics methods to efficiently validate a recent drug repurposing association of metformin with reduced cancer mortality. Methods By linking two large EHRs from Vanderbilt University Medical Center and Mayo Clinic to their tumor registries, we constructed a cohort including 32 415 adults with a cancer diagnosis at Vanderbilt and 79 258 cancer patients at Mayo from 1995 to 2010. Using automated informatics methods, we further identified type 2 diabetes patients within the cancer cohort and determined their drug exposure information, as well as other covariates such as smoking status. We then estimated HRs for all-cause mortality and their associated 95% CIs using stratified Cox proportional hazard models. HRs were estimated according to metformin exposure, adjusted for age at diagnosis, sex, race, body mass index, tobacco use, insulin use, cancer type, and non-cancer Charlson comorbidity index. Results Among all Vanderbilt cancer patients, metformin was associated with a 22% decrease in overall mortality compared to other oral hypoglycemic medications (HR 0.78; 95% CI 0.69 to 0.88) and with a 39% decrease compared to type 2 diabetes patients on insulin only (HR 0.61; 95% CI 0.50 to 0.73). Diabetic patients on metformin also had a 23% improved survival compared with non-diabetic patients (HR 0.77; 95% CI 0.71 to 0.85). These associations were replicated using the Mayo Clinic EHR data. Many site-specific cancers including breast, colorectal, lung, and prostate demonstrated reduced mortality with metformin use in at least one EHR. Conclusions EHR data suggested that the use of metformin was associated with decreased mortality after a cancer diagnosis compared with diabetic and non-diabetic cancer patients not on metformin, indicating its potential as a chemotherapeutic regimen. This study serves as a model for robust and inexpensive validation studies for drug repurposing signals using EHR data.