Fu, Sunyang

Background . Patients increasingly use asynchronous communication platforms to converse with care teams. Natural language processing (NLP) to classify content and automate triage of these messages has great potential to enhance clinical efficiency. We characterize the contents of a corpus of portal messages generated by patients using NLP methods. We aim to demonstrate descriptive analyses of patient text that can contribute to the development of future sophisticated NLP applications. Methods . We collected approximately 3,000 portal messages from the cardiology, dermatology, and gastroenterology departments at Mayo Clinic. After labeling these messages as either Active Symptom, Logistical, Prescription, or Update, we used NER (named entity recognition) to identify medical concepts based on the UMLS library. We hierarchically analyzed the distribution of these messages in terms of departments, message types, medical concepts, and keywords therewithin. Results . Active Symptom and Logistical content types comprised approximately 67% of the message cohort. The “Findings” medical concept had the largest number of keywords across all groupings of content types and departments. “Anatomical Sites” and “Disorders” keywords were more prevalent in Active Symptom messages, while “Drugs” keywords were most prevalent in Prescription messages. Logistical messages tended to have the lower proportions of “Anatomical Sites,”, “Disorders,”, “Drugs,”, and “Findings” keywords when compared to other message content types. Conclusions . This descriptive corpus analysis sheds light on the content and foci of portal messages. The insight into the content and differences among message themes can inform the development of more robust NLP models.

Abstract Background Delirium is underdiagnosed in clinical practice and is not routinely coded for billing. Manual chart review can be used to identify the occurrence of delirium; however, it is labor-intensive and impractical for large-scale studies. Natural language processing (NLP) has the capability to process raw text in electronic health records (EHRs) and determine the meaning of the information. We developed and validated NLP algorithms to automatically identify the occurrence of delirium from EHRs. Methods This study used a randomly selected cohort from the population-based Mayo Clinic Biobank (N = 300, age ≥65). We adopted the standardized evidence-based framework confusion assessment method (CAM) to develop and evaluate NLP algorithms to identify the occurrence of delirium using clinical notes in EHRs. Two NLP algorithms were developed based on CAM criteria: one based on the original CAM (NLP-CAM; delirium vs no delirium) and another based on our modified CAM (NLP-mCAM; definite, possible, and no delirium). The sensitivity, specificity, and accuracy were used for concordance in delirium status between NLP algorithms and manual chart review as the gold standard. The prevalence of delirium cases was examined using International Classification of Diseases, 9th Revision (ICD-9), NLP-CAM, and NLP-mCAM. Results NLP-CAM demonstrated a sensitivity, specificity, and accuracy of 0.919, 1.000, and 0.967, respectively. NLP-mCAM demonstrated sensitivity, specificity, and accuracy of 0.827, 0.913, and 0.827, respectively. The prevalence analysis of delirium showed that the NLP-CAM algorithm identified 12 651 (9.4%) delirium patients, the NLP-mCAM algorithm identified 20 611 (15.3%) definite delirium cases, and 10 762 (8.0%) possible cases. Conclusions NLP algorithms based on the standardized evidence-based CAM framework demonstrated high performance in delineating delirium status in an expeditious and cost-effective manner.

Update This article was updated on December 6, 2019, because of a previous error. On page 1936, in Table VII, “Performance of the Bearing Surface Algorithm,” the row that had read “Bearing surface predicted by algorithm” now reads “Bearing surface predicted by algorithm*.” An erratum has been published: J Bone Joint Surg Am. 2020 Jan 2;102(1):e4. Update This article was updated on March 31, 2020, because of a previous error. On page 1934, in Table IV (“THA Bearing Surface-Related Keywords in Operative Notes”), the row that had read “Femoral stem; stem; HFx-stem; femoral component; femoral component/stem; permanent prosthesis; stem fem cemented” now reads “Femoral head; ball; delta head; delta ceramic head; ion treated; BIOLOX delta; ceramic head; ceramic femoral head; ceramic offset head; ceramic (size) head; alumina ceramic head; alumina prosthetic head; alumna ceramic head; BIOLOX ceramic head; BIOLOX delta head; BIOLOX femoral head; BIOLOX delta ceramic head.” An erratum has been published: J Bone Joint Surg Am. 2020 May 6;102(9):e43. Background: Manual chart review is labor-intensive and requires specialized knowledge possessed by highly trained medical professionals. Natural language processing (NLP) tools are distinctive in their ability to extract critical information from raw text in electronic health records (EHRs). As a proof of concept for the potential application of this technology, we examined the ability of NLP to correctly identify common elements described by surgeons in operative notes for total hip arthroplasty (THA). Methods: We evaluated primary THAs that had been performed at a single academic institution from 2000 to 2015. A training sample of operative reports was randomly selected to develop prototype NLP algorithms, and additional operative reports were randomly selected as the test sample. Three separate algorithms were created with rules aimed at capturing (1) the operative approach, (2) the fixation method, and (3) the bearing surface category. The algorithms were applied to operative notes to evaluate the language used by 29 different surgeons at our center and were applied to EHR data from outside facilities to determine external validity. Accuracy statistics were calculated with use of manual chart review as the gold standard. Results: The operative approach algorithm demonstrated an accuracy of 99.2% (95% confidence interval [CI], 97.1% to 99.9%). The fixation technique algorithm demonstrated an accuracy of 90.7% (95% CI, 86.8% to 93.8%). The bearing surface algorithm demonstrated an accuracy of 95.8% (95% CI, 92.7% to 97.8%). Additionally, the NLP algorithms applied to operative reports from other institutions yielded comparable performance, demonstrating external validity. Conclusions: NLP-enabled algorithms are a promising alternative to the current gold standard of manual chart review for identifying common data elements from orthopaedic operative notes. The present study provides a proof of concept for use of NLP techniques in clinical research studies and registry-development endeavors to reliably extract data of interest in an expeditious and cost-effective manner.