Banerjee, Imon

Abstract Objective To develop a natural language processing (NLP) algorithm that can accurately extract headache frequency from free‐text clinical notes. Background Headache frequency, defined as the number of days with any headache in a month (or 4 weeks), remains a key parameter in the evaluation of treatment response to migraine preventive medications. However, due to the variations and inconsistencies in documentation by clinicians, significant challenges exist to accurately extract headache frequency from the electronic health record (EHR) by traditional NLP algorithms. Methods This was a retrospective cross‐sectional study with patients identified from two tertiary headache referral centers, Mayo Clinic Arizona and Mayo Clinic Rochester. All neurology consultation notes written by 15 specialized clinicians (11 headache specialists and 4 nurse practitioners) between 2012 and 2022 were extracted and 1915 notes were used for model fine‐tuning (90%) and testing (10%). We employed four different NLP frameworks: (1) ClinicalBERT (Bidirectional Encoder Representations from Transformers) regression model, (2) Generative Pre‐Trained Transformer‐2 (GPT‐2) Question Answering (QA) model zero‐shot, (3) GPT‐2 QA model few‐shot training fine‐tuned on clinical notes, and (4) GPT‐2 generative model few‐shot training fine‐tuned on clinical notes to generate the answer by considering the context of included text. Results The mean (standard deviation) headache frequency of our training and testing datasets were 13.4 (10.9) and 14.4 (11.2), respectively. The GPT‐2 generative model was the best‐performing model with an accuracy of 0.92 (0.91, 0.93, 95% confidence interval [CI]) and R 2 score of 0.89 (0.87, 0.90, 95% CI), and all GPT‐2–based models outperformed the ClinicalBERT model in terms of exact matching accuracy. Although the ClinicalBERT regression model had the lowest accuracy of 0.27 (0.26, 0.28), it demonstrated a high R 2 score of 0.88 (0.85, 0.89), suggesting the ClinicalBERT model can reasonably predict the headache frequency within a range of ≤ ± 3 days, and the R 2 score was higher than the GPT‐2 QA zero‐shot model or GPT‐2 QA model few‐shot training fine‐tuned model. Conclusion We developed a robust information extraction model based on a state‐of‐the‐art large language model, a GPT‐2 generative model that can extract headache frequency from EHR free‐text clinical notes with high accuracy and R 2 score. It overcame several challenges related to different ways clinicians document headache frequency that were not easily achieved by traditional NLP models. We also showed that GPT‐2–based frameworks outperformed ClinicalBERT in terms of accuracy in extracting headache frequency from clinical notes. To facilitate research in the field, we released the GPT‐2 generative model and inference code with open‐source license of community use in GitHub. Additional fine‐tuning of the algorithm might be required when applied to different health‐care systems for various clinical use cases.

Abstract Background The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. Methods We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. Results Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). Conclusions We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.

Abstract Purpose: There is a need to improve current risk stratification of stage II colorectal cancer to better inform risk of recurrence and guide adjuvant chemotherapy. We sought to examine whether integration of QuantCRC, a digital pathology biomarker utilizing hematoxylin and eosin–stained slides, provides improved risk stratification over current American Society of Clinical Oncology (ASCO) guidelines. Experimental Design: ASCO and QuantCRC-integrated schemes were applied to a cohort of 398 mismatch-repair proficient (MMRP) stage II colorectal cancers from three large academic medical centers. The ASCO stage II scheme was taken from recent guidelines. The QuantCRC-integrated scheme utilized pT3 versus pT4 and a QuantCRC-derived risk classification. Evaluation of recurrence-free survival (RFS) according to these risk schemes was compared using the log-rank test and HR. Results: Integration of QuantCRC provides improved risk stratification compared with the ASCO scheme for stage II MMRP colorectal cancers. The QuantCRC-integrated scheme placed more stage II tumors in the low-risk group compared with the ASCO scheme (62.5% vs. 42.2%) without compromising excellent 3-year RFS. The QuantCRC-integrated scheme provided larger HR for both intermediate-risk (2.27; 95% CI, 1.32–3.91; P = 0.003) and high-risk (3.27; 95% CI, 1.42–7.55; P = 0.006) groups compared with ASCO intermediate-risk (1.58; 95% CI, 0.87–2.87; P = 0.1) and high-risk (2.24; 95% CI, 1.09–4.62; P = 0.03) groups. The QuantCRC-integrated risk groups remained prognostic in the subgroup of patients that did not receive any adjuvant chemotherapy. Conclusions: Incorporation of QuantCRC into risk stratification provides a powerful predictor of RFS that has potential to guide subsequent treatment and surveillance for stage II MMRP colorectal cancers.

Abstract Percutaneous Coronary Intervention (PCI) is a widely utilized standard ofcare and efficacious intervention in the management of coronary artery disease,improving prognosis and symptoms for many patients. Prediction of post-PCIoutcomes contributes to effective patient management and quality improvementinitiatives in healthcare. Our study focuses on comparative performance analysisfor state-of-the-art vision transformer (ViT) and convolutional neural network(CNN) on multimodal data analysis in a joint fusion framework. We integrateimages of electrocardiogram (ECG) data and tabular electronic health records(EHR) to predict 3 clinically relevant post-PCI (6 months) endpoints - heart failure hospitalization, all-cause mortality, and ischemic stroke. To design a comparativemodel for ViT, we proposed a new joint fusion architecture that consistsof a convolutional neural network (CNN) with a convolutional block attentionmodule (CBAM). The learned representations are combined at an intermediatelayer followed by processing these representations using a fully connectedlayer. The proposed model demonstrates excellent performance with the highestAUROC scores of 0.849, 0.913, and 0.794 for predicting heart failure hospitalization,all-cause mortality, and stroke, respectively. Surpassing the baseline EHRmodel and ViT, the proposed CNN+CBAM fusion model showcases superiorpredictive capabilities for heart failure prediction.

162 Background: There is a need to improve current risk stratification of stage II and III colorectal cancer (CRC) to better inform risk of recurrence and guide adjuvant chemotherapy. The purpose of this study is to examine whether integration of QuantCRC, an AI-based digital pathology biomarker utilizing hematoxylin and eosin-stained slides, provides improved risk stratification over current American Society of Clinical Oncology (ASCO) guidelines. Methods: ASCO and QuantCRC-integrated risk schemes were applied to an observational cohort of 1,068 stage II and III CRCs. The stage II integrated scheme utilizes pT3 vs. pT4 and QuantCRC-derived risk groups. The stage III integrated scheme utilizes pT1-3 vs. pT4, pN1 vs. pN2, and QuantCRC-derived risk groups. Performance metrics included log-rank test, hazard ratios and Somers’ Dxy rank correlation. Results: Integration of QuantCRC provides improved risk stratification compared to the ASCO scheme for stage II and III CRC. The QuantCRC-integrated scheme placed more stage II tumors in the low-risk group compared to the ASCO scheme (69.3% vs. 60.4%) without decreased 3-year RFS. The QuantCRC-integrated scheme provided larger hazard ratios (HR) for both intermediate-risk (3.04, 95%CI 1.81-5.10, P=2.8x10-5) and high-risk (4.62, 95%CI 2.02-10.61, P=0.0003) groups compared to ASCO intermediate-risk (2.09, 95%CI 1.21-3.63, P=0.008) and high-risk (3.08, 95%CI 1.57-6.01, P=0.001) groups. The QuantCRC-integrated scheme for stage III tumors identified a small group of 80/518 (15.4%) CRCs at very high risk of recurrence with HR of 4.08 (95%CI 2.68-6.23, P=6.5x10-11) compared to a HR of 2.42 (95%CI 1.72-3.38, P=3.1x10-7) for 228/518 (44.0%) high-risk CRCs in the ASCO scheme. QuantCRC-integrated risk groups remained prognostic in stage III CRCs when stratified by presence or absence of any adjuvant chemotherapy. No difference in RFS were seen in QuantCRC-integrated low-risk and intermediate-risk stage III CRCs stratified by 3 vs. 6-months of oxaliplatin-based adjuvant chemotherapy suggesting that these two groups can be treated with 3-months of adjuvant therapy. Conclusions: Incorporation of QuantCRC into risk stratification provides a powerful predictor of RFS that has potential to guide subsequent treatment and surveillance.

In addition to the traditional clinical risk factors, an increasing amount of imaging biomarkers have shown value for cardiovascular risk prediction. Clinical and imaging data are captured from a variety of data sources during multiple patient encounters and are often analyzed independently. Initial studies showed that fusion of both clinical and imaging features results in superior prognostic performance compared with traditional scores. There are different approaches to fusion modeling, combining multiple data resources to optimize predictions, each with its own advantages and disadvantages. However, manual extraction of clinical and imaging data is time and labor intensive and often not feasible in clinical practice. An automated approach for clinical and imaging data extraction is highly desirable. Convolutional neural networks and natural language processing can be utilized for the extraction of electronic medical record data, imaging studies, and free-text data. This review outlines the current status of cardiovascular risk prediction and fusion modeling; and in addition gives an overview of different artificial intelligence approaches to automatically extract data from images and electronic medical records for this purpose.

Chest radiography (CXR) is the most frequently performed radiological test worldwide because of its wide availability, non-invasive nature, and low cost. The ability of CXR to diagnose cardiovascular diseases, give insight into cardiac function, and predict cardiovascular events is often underutilized, not clearly understood, and affected by inter- and intra-observer variability. Therefore, more sophisticated tests are generally needed to assess cardiovascular diseases. Considering the sustained increase in the incidence of cardiovascular diseases, it is critical to find accessible, fast, and reproducible tests to help diagnose these frequent conditions. The expanded focus on the application of artificial intelligence (AI) with respect to diagnostic cardiovascular imaging has also been applied to CXR, with several publications suggesting that AI models can be trained to detect cardiovascular conditions by identifying features in the CXR. Multiple models have been developed to predict mortality, cardiovascular morphology and function, coronary artery disease, valvular heart diseases, aortic diseases, arrhythmias, pulmonary hypertension, and heart failure. The available evidence demonstrates that the use of AI-based tools applied to CXR for the diagnosis of cardiovascular conditions and prognostication has the potential to transform clinical care. AI-analyzed CXRs could be utilized in the future as a complimentary, easy-to-apply technology to improve diagnosis and risk stratification for cardiovascular diseases. Such advances will likely help better target more advanced investigations, which may reduce the burden of testing in some cases, as well as better identify higher-risk patients who would benefit from earlier, dedicated, and comprehensive cardiovascular evaluation.