Liu, Hongfang

AbstractObjectivesTo create test collections for evaluating clinical information retrieval (IR) systems and advancing clinical IR research.Materials and MethodsElectronic health record (EHR) data, including structured and free-text data, from 45 000 patients who are a part of the Mayo Clinic Biobank cohort was retrieved from the clinical data warehouse. The clinical IR system indexed a total of 42 million free-text EHR documents. The search queries consisted of 56 topics developed through a collaboration between Mayo Clinic and Oregon Health & Science University. We described the creation of test collections, including a to-be-evaluated document pool using five retrieval models, and human assessment guidelines. We analyzed the relevance judgment results in terms of human agreement and time spent, and results of three levels of relevance, and reported performance of five retrieval models.ResultsThe two judges had a moderate overall agreement with a Kappa value of 0.49, spent a consistent amount of time judging the relevance, and were able to identify easy and difficult topics. The conventional retrieval model performed best on most topics while a concept-based retrieval model had better performance on the topics requiring conceptual level retrieval.DiscussionIR can provide an alternate approach to leveraging clinical narratives for patient information discovery as it is less dependent on semantics. Our study showed the feasibility of test collections along with a few challenges.ConclusionThe conventional test collections for evaluating the IR system show potential for successfully evaluating clinical IR systems with a few challenges to be investigated.

PURPOSEDrug development is becoming increasingly expensive and time consuming. Drug repurposing is one potential solution to accelerate drug discovery. However, limited research exists on the use of electronic health record (EHR) data for drug repurposing, and most published studies have been conducted in a hypothesis-driven manner that requires a predefined hypothesis about drugs and new indications. Whether EHRs can be used to detect drug repurposing signals is not clear. We want to demonstrate the feasibility of mining large, longitudinal EHRs for drug repurposing by detecting candidate noncancer drugs that can potentially be used for the treatment of cancer.PATIENTS AND METHODSBy linking cancer registry data to EHRs, we identified 43,310 patients with cancer treated at Vanderbilt University Medical Center (VUMC) and 98,366 treated at the Mayo Clinic. We assessed the effect of 146 noncancer drugs on cancer survival using VUMC EHR data and sought to replicate significant associations (false discovery rate < .1) using the identical approach with Mayo Clinic EHR data. To evaluate replicated signals further, we reviewed the biomedical literature and clinical trials on cancers for corroborating evidence.RESULTSWe identified 22 drugs from six drug classes (statins, proton pump inhibitors, angiotensin-converting enzyme inhibitors, β-blockers, nonsteroidal anti-inflammatory drugs, and α-1 blockers) associated with improved overall cancer survival (false discovery rate < .1) from VUMC; nine of the 22 drug associations were replicated at the Mayo Clinic. Literature and cancer clinical trial evaluations also showed very strong evidence to support the repurposing signals from EHRs.CONCLUSIONMining of EHRs for drug exposure–mediated survival signals is feasible and identifies potential candidates for antineoplastic repurposing. This study sets up a new model of mining EHRs for drug repurposing signals.

Background: Accurate identification of stroke cases from electronic health record (EHR) is needed for efficient and valid clinical and epidemiology research. A number of studies have evaluated the validity of using ICD-9 and ICD-10 codes for stroke identification. Previous research results show that sensitivity and positive predictive value (PPV) for stroke identification is the lowest among all other cerebrovascular diseases, with notable differences observed by stroke pathological subtypes. While most prevalent cerebrovascular disease cases can be detected using 430-438/I60-I69 collectively, more accurate and comprehensive stroke phenotyping algorithm is needed to identify incident stroke cases from EHR. In our work, we compared case identification results using ICD-codes exclusively with ICD plus a natural language processing (NLP) algorithm. Methods: We developed the NLP part of our stroke algorithm using a list of expert provided stroke-related keywords, which covers transient ischemic attack (TIA), ischemic stroke, and hemorrhagic stroke. We validated the hybrid (ICD+NLP) algorithm and compared it with ICD-exclusive algorithm in a previously established atrial fibrillation cohort (n=5,062). Clinical notes and ICD codes of all patients after the incident AF event were reviewed by two nurse abstracters to confirm new stroke incidences. All past clinical notes and ICD codes of a subset of patients (n=402) were reviewed to confirm lifetime (prior and current) stroke cases. Manual abstraction results were considered the gold standard for evaluation of ICD only and ICD+NLP automatic extraction. Sensitivity and positive predictive values (PPV) of both algorithms were calculated. Results: Among 5,062 patients, 593 patients were confirmed to have suffered a stroke after atrial fibrillation while 4,469 patients were confirmed with no EHR evidence of stroke after AF. The ICD-exclusive algorithm had a sensitivity of 47.2% and a PPV of 47.8% for detecting new stroke incidences. The hybrid stroke algorithm achieved a sensitivity of 92.4% and a PPV of 60.6% for extraction of incident stroke. For extraction of lifetime stroke cases, the hybrid approach achieved a sensitivity of 93.9% and a PPV of 88.7%. Performance of new stroke incidence extraction is limited because past/current stroke mentions in clinical notes are difficult to distinguish by our NLP algorithm (may be a future direction). Conclusions: We developed and validated a stroke algorithm that performed well for identifying incident and lifetime stroke cases. The addition of NLP into the stroke algorithm improved the sensitivity and PPV of compared to an ICD-exclusive algorithm.

Background Automated individualized risk prediction tools linked to electronic health records ( EHR s) are not available for management of patients with peripheral arterial disease. The goal of this study was to create a prognostic tool for patients with peripheral arterial disease using data elements automatically extracted from an EHR to enable real‐time and individualized risk prediction at the point of care. Methods and Results A previously validated phenotyping algorithm was deployed to an EHR linked to the Rochester Epidemiology Project to identify peripheral arterial disease cases from Olmsted County, MN, for the years 1998 to 2011. The study cohort was composed of 1676 patients: 593 patients died over 5‐year follow‐up. The c‐statistic for survival in the overall data set was 0.76 (95% confidence interval [CI], 0.74–0.78), and the c‐statistic across 10 cross‐validation data sets was 0.75 (95% CI, 0.73–0.77). Stratification of cases demonstrated increasing mortality risk by subgroup (low: hazard ratio, 0.35 [95% CI, 0.21–0.58]; intermediate‐high: hazard ratio, 2.98 [95% CI, 2.37–3.74]; high: hazard ratio, 8.44 [95% CI, 6.66–10.70], all P <0.0001 versus the reference subgroup). An equation for risk calculation was derived from Cox model parameters and β estimates. Big data infrastructure enabled deployment of the real‐time risk calculator to the point of care via the EHR . Conclusions This study demonstrates that electronic tools can be deployed to EHR s to create automated real‐time risk calculators to predict survival of patients with peripheral arterial disease. Moreover, the prognostic model developed may be translated to patient care as an automated and individualized real‐time risk calculator deployed at the point of care.