Banerjee, Imon

Echocardiography is an integral part of the diagnosis and management of cardiovascular disease. The use and application of artificial intelligence (AI) is a rapidly expanding field in medicine to improve consistency and reduce interobserver variability. AI can be successfully applied to echocardiography in addressing variance during image acquisition and interpretation. Furthermore, AI and machine learning can aid in the diagnosis and management of cardiovascular disease. In the realm of echocardiography, accurate interpretation is largely dependent on the subjective knowledge of the operator. Echocardiography is burdened by the high dependence on the level of experience of the operator, to a greater extent than other imaging modalities like computed tomography, nuclear imaging, and magnetic resonance imaging. AI technologies offer new opportunities for echocardiography to produce accurate, automated, and more consistent interpretations. This review discusses machine learning as a subfield within AI in relation to image interpretation and how machine learning can improve the diagnostic performance of echocardiography. This review also explores the published literature outlining the value of AI and its potential to improve patient care.

Aims:Increased left ventricular (LV) wall thickness is frequently encountered in transthoracic echocardiography (TTE). While accurate and early diagnosis is clinically important, given the differences in available therapeutic options and prognosis, an extensive workup is often required to establish the diagnosis. We propose the first echo-based, automated deep learning model with a fusion architecture to facilitate the evaluation and diagnosis of increased left ventricular (LV) wall thickness. Methods and Results: Patients with an established diagnosis of increased LV wall thickness (hypertrophic cardiomyopathy (HCM), cardiac amyloidosis (CA), and hypertensive heart disease (HTN)/others) between 1/2015 and 11/2019 at Mayo Clinic Arizona were identified. The cohort was divided into 80%/10%/10% for training, validation, and testing sets, respectively. Six baseline TTE views were used to optimize a pre-trained InceptionResnetV2 model. Each model output was used to train a meta-learner under a fusion architecture. Model performance was assessed by multiclass area under the receiver operating characteristic curve (AUROC). A total of 586 patients were used for the final analysis (194 HCM, 201 CA, and 191 HTN/others). The mean age was 55.0 years, and 57.8% were male. Among the individual view-dependent models, the apical 4-chamber model had the best performance (AUROC: HCM: 0.94, CA: 0.73, and HTN/other: 0.87). The final fusion model outperformed all the view-dependent models (AUROC: HCM: 0.93, CA: 0.90, and HTN/other: 0.92). Conclusion: The echo-based InceptionResnetV2 fusion model can accurately classify the main etiologies of increased LV wall thickness and can facilitate the process of diagnosis and workup.

AbstractBackgroundWhile low bone density is a major burden on US health system, current osteoporosis screening guidelines by the US Preventive Services Task Force are limited to women aged ≥65 and all postmenopausal women with certain risk factors. Even within recommended screening groups, actual screening rates are low (<26%) and vary across socioeconomic groups. The proposed model can opportunistically screen patients using abdominal CT studies for low bone density who may otherwise go undiagnosed.PurposeTo develop an artificial intelligence (AI) model for opportunistic screening of low bone density using both contrast and non‐contrast abdominopelvic computed tomography (CT) exams, for the purpose of referral to traditional bone health management, which typically begins with dual energy X‐ray absorptiometry (DXA).MethodsWe collected 6083 contrast‐enhanced CT imaging exams paired with DXA exams within ±6 months documented between May 2015 and August 2021 in a single institution with four major healthcare practice regions. Our fusion AI pipeline receives the coronal and axial plane images of a contrast enhanced abdominopelvic CT exam and basic patient demographics (age, gender, body cross section lengths) to predict risk of low bone mass. The models were trained on lumbar spine T‐scores from DXA exams and tested on multi‐site imaging exams. The model was again tested in a prospective group (N = 344) contrast‐enhanced and non‐contrast‐enhanced studies.ResultsThe models were evaluated on the same test set (1208 exams)—(1) Baseline model using demographic factors from electronic medical records (EMR) ‐ 0.7 area under the curve of receiver operator characteristic (AUROC); Imaging based models: (2) axial view ‐ 0.83 AUROC; (3) coronal view‐ 0.83 AUROC; (4) Fusion model—Imaging + demographic factors ‐ 0.86 AUROC. The prospective test yielded one missed positive DXA case with a hip prosthesis among 23 positive contrast‐enhanced CT exams and 0% false positive rate for non‐contrast studies. Both positive cases among non‐contrast enhanced CT exams were successfully detected. While only about 8% patients from prospective study received a DXA exam within 2 years, about 30% were detected with low bone mass by the fusion model, highlighting the need for opportunistic screening.ConclusionsThe fusion model, which combines two planes of CT images and EMRs data, outperformed individual models and provided a high, robust diagnostic performance for opportunistic screening of low bone density using contrast and non‐contrast CT exams. This model could potentially improve bone health risk assessment with no additional cost. The model's handling of metal implants is an ongoing effort.

Abstract Purpose Accurately predicting the clinical breast cancer subtypes could be extremely helpful for radiologists, pathologists, surgeons, and clinicians and inform future treatment prediction algorithms. Therefore, we evaluate and compare the accuracy of radiomic features extracted from contrast enhanced mammography (CEM) and magnetic resonance imaging (MRI) scans to make predictions to subtypes of breast cancer. Methods This HIPAA-compliant prospective single institution study was approved by the local institutional review board with written informed consent. Women with breast tumors 2 cm or larger underwent dynamic contrast-enhanced MRI and/or CEM for surgical staging. Semi-manual regions of interest were drawn by radiologist using Cancer Imaging Phenomics Toolkit (CaPTk). Radiomic features were obtained using PyRadiomics and MR- and CEM- based classification models were built on a low-dimensional representation of the features obtained via kernel principal component analysis. We subscribed to an ensemble tree-based learning approach called extremely randomized trees (ERT) to predict cancer subtypes captured via immunohistochemistry markers. Results For MRI analysis, 124 women with newly diagnosed breast cancer were included in the analysis comprising 49 HR+HER2-, 37 HR+HER2+, 11 HR-HER2+, and 27 HR-HER2- cases. For CEM analysis, models were built using data from 170 female patients including 74 HR+HER2-, 41 HR+HER2+, 14 HR-HER2+, and 43 HR-HER2-. CEM based model resulted in accuracies of 55%, 72%, 88%, and 71% respectively for HR+HER2-, HR+HER2+, HR-HER2+, and HR-HER2- whereas MRI based model alone led to accuracies of 54%, 62%, 89%, and 76% respectively for HR+HER2-, HR+HER2+, HR-HER2+, and HR-HER2-. Conclusions Radiomic features extracted from CEM and MR were strong predictors of breast cancer subtypes with CEM-based radiomic features performing slightly better, though not statistically significantly better (p = 0.82), than its MRI counterpart.

Abstract Background. Clinical medicine relies heavily on the synthesis of information and data from multiple sources. However, often simple feature concatenation is used as a strategy for developing a multimodal machine learning model in the cardiovascular domain, and thus the models are often limited by pre-selected features and moderate accuracy. Method. We proposed a two-branched joint fusion model for fusing the 12-lead electrocardiogram (ECG) signal data with clinical variables from the electronic medical record (EMR) in an end-to-end deep learning architecture. The model follows the joint fusion scheme and learns complementary information from ECG and EMR. Retrospective data from the Mayo Clinic Health Systems across four sites for patients that underwent percutaneous coronary intervention (PCI) were obtained. Model performance was assessed by area under the receiver-operating characteristics (AUROC) and Delong’s test. Results. The final cohort included 17,356 unique patients with a mean age of 67.2 ± 12.6 year (mean ± std) and 9,163 (52.7%) were male. The joint fusion model outperformed the ECG time-domain model with statistical margin. The model with clinical data obtained the highest AUROC for all-cause mortality (0.91 at 6 months) but the joint fusion model outperformed for cardiovascular outcomes - heart failure hospitalization and ischemic stroke with a significant margin (Delong’s p < 0.05). Conclusion. To the best of our knowledge, this is the first study that developed a deep learning model with joint fusion architecture for the prediction of post-PCI prognosis and outperformed machine learning models developed using traditional single-source features (clinical variables or ECG features). Adding ECG data with clinical variables did not improve prediction of all-cause mortality as may be expected, but the improved performance of related cardiac outcomes shows that the fusion of ECG generates additional value.