Barve, Rakesh

11176 Background: Progression-free survival (PFS) is frequently measured in oncology clinical trials. In analyses outside of the trial setting, various strategies have been utilized to assess real-world PFS (rwPFS), including manual abstraction of clinical records, natural language processing (NLP) of oncology notes and radiology reports, and longitudinal analyses of radiologic images. Here we develop and validate a new semi-automated workflow that combines NLP of clinical notes with structured electronic health record data to facilitate the evaluation of rwPFS in patients with metastatic breast cancer (mBC). Methods: This study analyzes de-identified EHR data using nference nSights. The data is extracted following privacy-preserving protocols and is HIPAA compliant. The overall cohort included 316 patients with HR-positive, HER2-negative mBC who initiated Palbociclib and Letrozole combination therapy between January 1, 2015 and December 31, 2021. We developed and implemented an ensemble of deep-learning NLP frameworks to capture progression events from unstructured clinical notes and radiology reports. A change in the line of therapy, as determined by structured drug order/administration records, was also considered a progression event. We used manually curated “ground-truth” datasets to evaluate the performance of the progression-event capture workflow at the levels of both sentences (N = 1000) and patients (N = 100) by calculating sensitivity, specificity, precision, accuracy, and F1 scores. Progression events and censoring events (death, loss to follow-up, end of study period) were considered to compute rwPFS. Results: At the sentence level, progression events were captured from clinical notes and radiology reports with a sensitivity of 99.8%, specificity of 96.7%, and accuracy of 98.2% (Table). At the patient level, initial progression was correctly captured within a window of +/-30 days with a sensitivity of 92.5%, specificity of 83.0%, and accuracy of 88.0% (Table). In a sample of 100 patients, the median rwPFS was determined to be 25 months (95% CI; 15-35 months) by manual curation and 22 months (95% CI; 15-35 months) by the semi-automated workflow. In the overall cohort, median rwPFS was 20 months (95% CI; 18-25 months). Conclusions: An ensemble of NLP algorithms extracted progression events from clinical notes and radiology reports with high accuracy. A semi-automated workflow enabled rapid and accurate determination of rwPFS in mBC patients receiving a combination chemotherapy regimen. Further evaluation of this workflow to estimate rwPFS in other cancers and therapeutic settings is warranted. [Table: see text]

Introduction: Artificial intelligence applied to electrocardiograms (ECG-AI) enables early and efficient detection of cardiovascular disease. The technical and clinical impact of different approaches to designing ECG-AI remains under-explored. We consider left ventricular hypertrophy (LVH) and examine the impact of disease-specific labeling of minority etiologies of left ventricular wall thickening- hypertrophic cardiomyopathy (HCM) and cardiac amyloidosis (CA) - on ECG-AI performance. Hypothesis: ECG-AI designed to detect HCM or CA outperforms ECG-AI broadly trained to identify LVH due to differences in attention to key ECG features associated with HCM or CA. Methods: Independent neural networks were trained to identify HCM, CA, and LVH. HCM and CA patients were identified using the full depth of EHR data and validated via clinician adjudication (N = 3,025 and 4,251 patients, respectively). Moderate and severe LVH patients (N = 98,144) were identified using echocardiogram results and international LVH criteria. We compared the ability of disease-specific ECG-AI vs. LVH ECG-AI to detect HCM (or CA) by examining model performance within each of the three test sets. Results: 52.6% of HCM patients and 49.5% of CA patients met LVH criteria. Within the HCM test set, the AUROC, sensitivity, specificity, and positive predictive value (PPV) of the HCM ECG-AI were 0.94, 0.88, 0.85, and 60%, respectively, whereas the same for LVH ECG-AI were 0.83, 0.83, 0.64, and 37%. Within the CA test set, CA ECG-AI achieved AUROC, sensitivity, specificity, and PPV of 0.94, 0.81, 0.91, and 69%, respectively, whereas the same for LVH ECG-AI were 0.75, 0.73, 0.68, and 31%. ECGs positive for HCM exhibited higher P wave area, QRS deflection and R wave amplitude compared to LVH-positive ECGs (Cohen's D > 0.75). ECGs positive for CA exhibited lower R wave amplitude/area, QRS deflection and QRS areas (Cohen's D > 0.6). Conclusion: Models trained using carefully labeled HCM or CA identify distinct signals associated with the diseases, resulting in improved disease discrimination compared to a model trained to identify echocardiographic left ventricular thickening. Labeling choices should be communicated transparently to clinicians to ensure safe and effective use of ECG-AI.

Background: Early diagnosis of heart failure with reduced ejection fraction (HFrEF) can lead to reduced healthcare costs and improved outcomes. In this study, we evaluated the economic effectiveness of an AI algorithm for detecting low ejection fraction using 12-lead ECG data. Aims: The primary aim was to estimate the health economic impact of the AI-ECG algorithm intervention to detect low ejection fraction. Methods: The study population consisted of 113,719 individuals without a prior history of HFrEF who visited a multistate health system between 2010-2018 with 12-lead ECG data and an echocardiogram within 1 year. The AI-ECG algorithm was run retrospectively, and projected costs were estimated assuming each patient incurred the average cost based on their age group and AI-detected heart failure stage. Hospitalization costs were estimated using Medicare reimbursement rates, and economic benefits from increased survival were estimated using a willingness to pay of $50,000 per quality-adjusted life year (QALY) gained. Results: At the optimal model threshold, the total cost savings per patient (including benefits from QALYs gained) for the AI-ECG algorithm vs. standard-of-care on the overall study population were projected to be $79.11 (95% CI: [$53.77, $104.44]) and $124.93 (95% CI: [$83.99, $165.88]) for the 2 and 4 year periods, respectively. For patients over 50 years old, the projected total cost savings were $85.56 (95% CI: [$56.35, $114.78]) and $135.28 (95% CI: [$88.18, $182.38]) per patient for the 2 and 4 year periods, respectively. See Figure. Conclusions: The AI-ECG algorithm to detect low ejection fraction is a cost-effective early screening tool for HFrEF.

Understanding temporal dynamics of COVID-19 symptoms could provide fine-grained resolution to guide clinical decision-making. Here, we use deep neural networks over an institution-wide platform for the augmented curation of clinical notes from 77,167 patients subjected to COVID-19 PCR testing. By contrasting Electronic Health Record (EHR)-derived symptoms of COVID-19-positive (COVID pos ; n = 2,317) versus COVID-19-negative (COVID neg ; n = 74,850) patients for the week preceding the PCR testing date, we identify anosmia/dysgeusia (27.1-fold), fever/chills (2.6-fold), respiratory difficulty (2.2-fold), cough (2.2-fold), myalgia/arthralgia (2-fold), and diarrhea (1.4-fold) as significantly amplified in COVID pos over COVID neg patients. The combination of cough and fever/chills has 4.2-fold amplification in COVID pos patients during the week prior to PCR testing, in addition to anosmia/dysgeusia, constitutes the earliest EHR-derived signature of COVID-19. This study introduces an Augmented Intelligence platform for the real-time synthesis of institutional biomedical knowledge. The platform holds tremendous potential for scaling up curation throughput, thus enabling EHR-powered early disease diagnosis.