Abstract Background Artificial intelligence enhanced electrocardiograms (AI-ECG) have matured over the past 10 years. More recently, the AI-ECG estimate of biological sex has shown potential as a novel tool for cardiovascular risk stratification among women. Cardiovascular disease is a leading cause of pregnancy-related mortality; however, there have been limited approaches for estimating the physiological demands of pregnancy and its impact of maternal cardiovascular outcomes. Purpose To estimate pregnancy-related changes on AI-ECG-based sex and age estimation in a prospective cohort of otherwise healthy, complication free pregnant women. Methods This secondary analysis of the SPEC-AI Nigeria Clinical Trial identified pregnant women without any pregnancy-related complications (at baseline or during follow-up) who had at least two 12 lead ECGs recorded during the study. The ECGs were analyzed with previously-developed algorithms to predict age and biological sex. To address bias in the AI-ECG age estimate, a reference cohort of 25,144 healthy patients was analyzed using quantile regression to estimate the 75th percentile of the reference distribution as a function of age. The estimate of biological sex was used to determine a sex discordance score with a discordance of 0.2 considered clinically relevant. To simplify the trends in the data, the ECGs were categorized as during pregnancy (<=28 days prior to birth), peripartum (+/- 28 days of birth), and postpartum (28+ days after birth), and mixed effects models with a random intercept were used for analysis. Results A total of 207 females with a median age of 31 years (IQR 28 to 36 years) and normal blood pressures (systolic: 110 (IQR 10 to 120); diastolic: 70 (IQR 60-77); mmHg) were included from 5 clinical sites in Nigeria. The majority (73%, 150/207) of participants were in their second trimester at time of enrollment and the median number of ECGs per participant was 3 (max 7). The sex discordance score was elevated in 38% of the women at baseline and this percentage fell to 23% postpartum; which resulted in a decrease in the mean discordance score of 0.03 (95% CI: -0.06 to 0.0; p=0.028). Conversely, the percentage of patients with elevated age prediction increased from 35% to 41% over the course of pregnancy. This resulted in an increase in the bias-corrected AI-ECG age of 3.0 years (95% CI: 1.5 to 4.6 years; p<0.001). Conclusion This study demonstrates the preliminary utility of serial AI-ECG based assessment as a marker for the biological effects of pregnancy. The increase in sex discordance score during pregnancy may be related to changes in cardiac structure, such as increased ventricular mass and chamber dilation which often reverses postpartum. However, the impact on age estimation appeared to persist in the early postpartum period. Further study is warranted to associate any changes observed with long term maternal or fetal outcomes.Figure
Clinical impact of discordant stress test/angiography results: a sex-stratified analysis
Abstract Background Ischemic heart disease is the leading cause of death among women. Non-obstructive coronary disease (NOCAD), often identified by a positive stress test and negative coronary angiogram result, is linked to adverse outcomes, particularly among women. Purpose This study evaluated the impact of discordant Single Photon Emission Computed Tomography (SPECT) Myocardial Perfusion Imaging (MPI) stress test and coronary angiogram results on repeat invasive angiography and survival, while assessing sex-related differences. Methods Data were extracted from a single health system for patients who underwent a SPECT MPI stress test, and invasive coronary angiography performed within 3 months from September 2002 to July 2018. Discordance was defined as: Profile A – a positive stress test with negative angiography (epicardial coronary vessel stenosis <50%); Profile B – a negative stress test with positive angiography (at least one epicardial coronary vessel stenosis >=50%). To assess the association between discordance and all-cause mortality, Cox-proportional hazards regression was used to analyze the most recent stress/angiogram pair (adjusting for race, age, and number of prior angiograms). For repeat invasive angiography, where mortality was considered a competing event, the Fine-Gray proportional hazards model for the sub-distribution of a competing risk was used to analyze the first stress/angiogram pair (adjusting for race and age). Hazard ratios (HRs), sub-distribution hazard ratios (SHRs), and corresponding 95% confidence intervals (CIs) were reported. Results 5,179 were included in the final analysis, median age was 67.9 years and 32.7% were female. Approximately 15% and 24% were classified as Profile A and B respectively. Patients with a negative stress/negative angiogram were designated as the reference group. Profile A was not significantly associated with mortality or repeat invasive testing in both sexes. Profile B was associated with the highest risk of death (female aHR: 1.58, P<.001; male aHR: 1.54, P<.001) followed by those with a positive stress/positive angiogram (female aHR: 1.39, P=.009; male aHR: 1.25, P=.05) compared to the reference group. Profile B was significantly associated with repeat invasive testing and a stronger association was seen in females (female aSHR: 9.84, P< .001; male aSHR: 3.42, P<.001). A similar pattern was seen for those with a positive stress/positive angiogram. Conclusion Discordant Profile A showed no significant association with mortality or repeat invasive testing. Discordant Profile B was associated with the highest mortality risk in both sexes and showed a stronger association with repeat invasive testing among females. These findings underscore the importance of careful patient selection for stress testing, the need for improved stress testing options (especially in females), and prompt coronary evaluation in patients with a negative stress test and persistent symptoms or high-risk features.
Long-term cellular and humoral responses to SARS-CoV-2 vaccinations in patients with solid malignancies undergoing chemotherapy
Background While SARS-CoV-2 vaccines are highly effective in healthy individuals, the magnitude and durability of humoral and cellular responses in patients with solid malignancies receiving chemotherapy remain understudied. Previous reports suggest that patients with cancer may not mount adequate immune response after SARS-CoV-2 vaccination. Additionally, most studies focused on humoral responses, while data regarding cellular immune responses are scarce. In this study, we evaluated humoral and cellular responses in patients with solid tumors receiving chemotherapy compared to healthy individuals up to one year after vaccinations. Methods Patients aged ≥18 who were willing to receive the SARS-CoV-2 vaccine were enrolled. Anti-SARS-CoV-2 immunoassays were used to detect antibodies against nucleocapsid and spike proteins. Human IFN-γ Fluorospot assay was used to determine antigen−specific T-cell responses. Data were compared between groups using Mann−Whitney test for continuous variables and Fisher’s exact test for categorical variables. Results A total of 67 subjects (47 patients with cancer and 20 healthy individuals) were included. 1–3 months following the second vaccine doses, 96% of patients with cancer and 100% of healthy individuals demonstrated a positive humoral response. While the positivity rate was not significantly different, patients with cancer had significantly lower spike IgG antibodies than healthy individuals. However, this difference diminished at 6 months when patients with cancer had increased antibodies compared to decreased antibodies in the healthy cohort and no difference was noticed at 12-month. Patients with cancer developed a similar antigen-specific T-cell response as healthy individuals at 1–3 months, 6 months and 12 months. There were no significant differences when comparing patients aged ≤ 55 years vs. >55 years, stages I-III vs. IV, single vs. multiple chemotherapy, and BNT162b2 vs. mRNA-1273 vaccines. There was a significant moderate correlation between neutralization and antibody levels at 12 months. However, despite patients with cancer having a significantly higher COVID risk score, there were no significant differences in COVID-19 infection and hospitalization between patients with cancer and healthy individuals. Conclusion Despite initial impaired antibody responses to SARS-CoV-2 vaccinations, patients with solid malignancies receiving chemotherapy effectively generated long-term cellular and humoral responses by 6 and 12 months, leading to similar infection and hospitalization rates compared to healthy individuals.
Discovery of an Artificial Intelligence Label Feedback Loop: How the Success of a Clinically Implemented Artificial Intelligence Algorithm Has Created an Unforeseen Challenge to Algorithm Retraining
Abstract Background Artificial intelligence (AI) augmented laboratory tests can improve quality, efficiency, cost-effectiveness, and staff satisfaction. However, the clinical success of these tests can introduce unforeseen challenges for model retraining. This study describes the discovery of an “AI label feedback loop” in a clinically implemented AI-augmented kidney stone composition test. Methods An AI-augmented kidney stone composition test (V1) has been previously deployed for clinical kidney stone characterization. After several years of clinical use, a retrained model (V2) was developed using 6 times more data. Model performance of both V1 and V2 were evaluated across 3 datasets: a recent production validation (hold-out) set (mostly AI-influenced labels), the original V1 validation set (pre-AI, entirely human-labeled), and a subset of recent cases with exclusively human-generated or human-corrected labels. Results V2 demonstrated a 10% lower concordance rate than V1 when evaluated on the recent production hold-out set, despite a much larger training dataset. Performance between V1 and V2 was similar when applied to the pre-AI validation set. Notably, V2 outperformed V1 on the recent subset of cases with human-only or human-corrected labels, particularly for less-common stone types. These findings revealed an AI label feedback loop, confounding retraining and evaluation. Conclusions The integration of AI into clinical practice can potentially influence reported test results, complicating the development and evaluation of future models. To mitigate AI label feedback loops, ongoing human annotation and careful validation set construction are essential. These strategies can ensure reliable performance assessment and support the safe evolution of clinical AI systems.
Abstract 4367166: A Multicenter Study of Detection of Pulmonary Hypertension Based on Point-of-Care 12- Lead ECG Data
Background: Pulmonary hypertension (PH) is a life-threatening disease affecting up to 1% of the global population. Diagnosis can be challenging and is often delayed due to the need for advanced imaging and invasive procedures. The use of artificial intelligence applied to ECGs (ECG-AI) has been shown to detect subtle patterns in voltage-time data and may be a valuable tool for the early detection of PH. Hypothesis and Purpose: To evaluate the performance of a previously trained, ECG-AI algorithm to detect PH (ECG-AI PH) using real-world data (RWD) collected in a multicenter, validation study. A joint primary hypothesis required sensitivity (Sn), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) to exceed the null values of 76%, 70%, 10% and 90%. Study Design and Methods: This retrospective validation study was conducted at 5 geographically diverse U.S. health systems. Adult subjects were eligible for inclusion if they had a 12-lead ECG paired with an echocardiogram (Echo) in which tricuspid regurgitation velocity (TRV) was recorded, following presentation with dyspnea. Patients were classified according to echocardiographic criteria as either PH (PH+, TRV >3.4 m/s) or controls (PH-, TRV ≤2.8 m/s) to simulate the real-world use of ECG-AI, where a positive result could lead to a follow-up Echo. The study database was locked before processing the digital ECGs with ECG-AI PH. Performance was also estimated in a subset of subjects that later had a right heart catheterization using mPAP ≥ 20mmHg as the definition of PH. Results: A total of 14281 subjects (53% female, 63% aged 50+ years) met the inclusion criteria, including 3019 PH+ cases and 11262 PH- controls (Figure). Sn and Sp were 84.0% (95% CI: 82.6%, 85.3%) and 71.7% (95% CI: 70.9%, 72.6%), respectively. The positive and negative predictive values were 18.9% (95% CI: 18.4%, 19.4%) and 98.3% (95% CI: 98.1%, 98.4%), respectively, at 7.3% prevalence. Each endpoint met pre-defined performance criteria. In the subset of 1683 subjects with a follow up RHC, performance remained robust (Sn 85% (1155/1358); Sp 46% (151/325)). Conclusion: While ECG-AI PH was first developed as an investigational tool to detect PH, continued development as software as a medical device for clinical use demonstrated that the algorithm retained strong performance to detect PH in diverse, non-overlapping clinical settings and patient populations.
Genome-wide association study of neuropathological features in Lewy body disease
Abstract Studies assessing genetic associations with neuropathological features in Lewy body disease (LBD) have been limited to candidate gene investigations, and therefore, information is lacking regarding the genetic architecture of the neuropathology of LBD. In the current study, we examined a large series of neuropathologically confirmed LBD cases (n = 980 in the discovery series, n = 503 in the replication series) and performed genome-wide association studies of 11 different neuropathological outcome measures. The 11 neuropathological outcomes included Braak neurofibrillary tangle (NFT) stage, Thal amyloid phase, LBD subtype, Lewy body (LB) counts in five different brain regions, dorsolateral and ventromedial putaminal tyrosine hydroxylase immunoreactivity and neuronal loss in the ventrolateral part of the substantia nigra. Associations between variants and outcomes were assessed using regression models appropriate for the nature of the given neuropathological outcome and that were adjusted for age at death, sex and top principal components of genetic data. In the discovery series, APOE rs429358 (i.e. APOE ε4) was associated with a greater severity of each of Braak NFT stage [odds ratio (OR) = 3.07, P = 2.34 × 10−32], Thal amyloid phase (OR = 3.57, P = 3.28 × 10−29) and LBD subtype (OR = 1.78, P = 9.85 × 10−9), with similar findings observed in the independent replication series (Braak NFT stage, OR = 2.30, P = 2.70 × 10−11; Thal amyloid phase, OR = 3.17, P = 6.39 × 10−18; LBD subtype, OR = 2.68, P = 3.85 × 10−10). In the subgroup of cases with lower levels of Alzheimer's disease pathology (Braak NFT stage ≤ III and Thal amyloid phase ≤2), there was a strong association between APOE rs429358 and LBD subtype even when adjusting for Braak stage and Thal phase in the discovery series (n = 218, OR = 2.47, P = 0.007) and the replication series (n = 141, OR = 3.60, P = 0.006). Although additional genome-wide significant associations were identified in the discovery series between LINC01581/MCTP2 rs547411734 and lower middle frontal LB counts, between TLE3 rs3743309 and lower cingulate LB counts, and between GRIN2A/ATF7IP2 rs1097915 and lower parahippocampal LB counts, these findings were not observed in the replication series. Our results indicate that the APOE ε4 allele is the most prominent genetic determinant of severity of neuropathology in LBD. These findings represent a key step forward in our understanding of genetic drivers of neuropathological features in LBD. Future studies utilizing meta-analytical approaches will be important to more precisely assess other associations that were not quite genome-wide significant in the discovery series.
AI-Augmented Kidney Stone Composition Analysis with Auto-Release Improves Quality, Efficiency, Cost–Effectiveness, and Staff Satisfaction
Abstract Background We sought to evaluate key performance indicators related to an internally developed and deployed artificial intelligence (AI)-augmented kidney stone composition test system for potential improvements in test quality, efficiency, cost-effectiveness, and staff satisfaction. Methods We compared quality, efficiency, staff satisfaction, and financial data from the 6 months after the AI-augmented laboratory test system was deployed (test period) with data from the same 6-month period in the previous year (control period) to determine if AI-augmentation improved key performance indicators of this laboratory test. Results In the 6 months following the deployment (test period) of the AI-augmented kidney stone composition test system, 44 830 kidney stones were analyzed. Of these, 92% of kidney stones were eligible for AI-assisted interpretation. Out of these AI-eligible stones, 45% were able to be auto-released by the AI-augmented test system without human secondary review. Furthermore, the new AI-augmented kidney stone test system resulted in an apparent 40% reduction in incorrect laboratory results. Additionally, the new AI-augmented test system improved laboratory efficiency by 20%, improved staff satisfaction, and reduced the average analysis cost per kidney stone by $0.23. Conclusions The AI-augmented test system improved test quality, efficiency, cost-effectiveness and staff satisfaction related to this kidney stone composition test.
Diagnosing Allergic Contact Dermatitis Using Deep Learning: Single-Arm, Pragmatic Clinical Trial with an Observer Performance Study to Compare Artificial Intelligence Performance with Human Reader Performance
Abstract: Background: Allergic contact dermatitis is a common, pruritic, debilitating skin disease, affecting at least 20% of the population. Objective: To prospectively validate a computer vision algorithm across all Fitzpatrick skin types. Methods: Each participant was exposed to 10 allergens. The reference criterion was obtained 5 days after initial patch placement by a board-certified dermatologist. The algorithm processed photographs of the test site obtained on Day 5. Human performance in reading the photographs was also evaluated. Results: A total of 206 evaluable participants [mean age 39 years, 66% (136/206) female, and 47% with Fitzpatrick skin types IV–VI] completed testing. Forty-two percent (87/206) of participants experienced 1 or more allergic reaction resulting in a total of 132 allergic reactions. The model provided high discrimination (AUROC 0.86, 95% CI: 0.82–0.90) and specificity (93%, 95% CI: 92%–94%) but with lower sensitivity (58%, 95% CI: 49%–67%). Human performance interpreting the photographs ranged from providing similar performance to the algorithm to providing superior performance when combined across readers. There were no serious adverse events. Conclusions: The combination of a smartphone capture of patch testing sites with deep learning yielded high discrimination across a diverse sample.
143 Long-term cellular and humoral responses to SARS-CoV-2 vaccinations in patients with solid malignancies undergoing chemotherapy
Abstract Aims Recently, deep learning artificial intelligence (AI) models have been trained to detect cardiovascular conditions, including hypertrophic cardiomyopathy (HCM), from the 12-lead electrocardiogram (ECG). In this external validation study, we sought to assess the performance of an AI-ECG algorithm for detecting HCM in diverse international cohorts. Methods and results A convolutional neural network-based AI-ECG algorithm was developed previously in a single-centre North American HCM cohort (Mayo Clinic). This algorithm was applied to the raw 12-lead ECG data of patients with HCM and non-HCM controls from three external cohorts (Bern, Switzerland; Oxford, UK; and Seoul, South Korea). The algorithm’s ability to distinguish HCM vs. non-HCM status from the ECG alone was examined. A total of 773 patients with HCM and 3867 non-HCM controls were included across three sites in the merged external validation cohort. The HCM study sample comprised 54.6% East Asian, 43.2% White, and 2.2% Black patients. Median AI-ECG probabilities of HCM were 85% for patients with HCM and 0.3% for controls (P < 0.001). Overall, the AI-ECG algorithm had an area under the receiver operating characteristic curve (AUC) of 0.922 [95% confidence interval (CI) 0.910–0.934], with diagnostic accuracy 86.9%, sensitivity 82.8%, and specificity 87.7% for HCM detection. In age- and sex-matched analysis (case–control ratio 1:2), the AUC was 0.921 (95% CI 0.909–0.934) with accuracy 88.5%, sensitivity 82.8%, and specificity 90.4%. Conclusion The AI-ECG algorithm determined HCM status from the 12-lead ECG with high accuracy in diverse international cohorts, providing evidence for external validity. The value of this algorithm in improving HCM detection in clinical practice and screening settings requires prospective evaluation.