Halamka, John

Introduction: Asymptomatic LV systolic dysfunction is often underdiagnosed, leading to missed opportunities for treatment. In a recently completed pragmatic RCT, an AI algorithm performed on the ECGs ordered as part of routine practice increased early diagnosis of low EF. The current study aimed to understand whether the effectiveness of the AI algorithm integrated into routine care differed by the socioeconomic status of the patient. Methods: The main findings of the RCT were previously published. Clinicians in the intervention group had access to AI results (i.e., a positive or negative screening result, with positive results indicating a high likelihood of low EF). When a positive result was found, clinicians in the intervention group received an alert that encouraged them to order TTE if considered clinically appropriate. Clinicians in the control group did not have access or received alerts. Area deprivation index (ADI) was used to measure area-level social determinants of health. This study included 16,992 patients (control 8,225; intervention: 8,767) who had ECG performed for any indications between 8/5/2019 and 3/31/2020, no prior HF, and a valid address to derive ADI. The primary endpoint was a new diagnosis of EF≤50% within 3 months. Results: The mean age was 60.5±17.6 y. AI increased the diagnosis of low EF in the overall cohort (OR 1.42 [1.13, 1.78], p=0.01). The magnitude of benefit was greater in patients with higher socioeconomic status, e.g., living in an area that is less deprived (OR 1.88 [1.16, 3.05]), with higher median income, lower poverty rate, higher education level, better housing conditions, and higher proportion of white-collar workers, although the interaction was not statistically significant (Figure). Conclusions: AI enables early diagnosis of low EF and the magnitude of benefit might be numerically greater in patients with higher socioeconomic status, possibly due to variations in access to or referral for subsequent testing.

Importance Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed. Observations While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings. Conclusion and Relevance The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

We are at a pivotal moment in the development of healthcare artificial intelligence (AI), a point at which enthusiasm for machine learning has not caught up with the scientific evidence to support the equity and accuracy of diagnostic and therapeutic algorithms. This proposal examines algorithmic biases, including those related to race, gender and socioeconomic status, and accuracy, including the paucity of prospective studies and lack of multisite validation. We then suggest solutions to these problems. We describe the Mayo Clinic, Duke University, Change Healthcare project that is evaluating 35.1 billion healthcare records for bias. And we propose ‘Ingredients’ style labels and an AI evaluation/testing system to help clinicians judge the merits of products and services that include algorithms. Said testing would include input data sources and types, dataset population composition, algorithm validation techniques, bias assessment evaluation and performance metrics.

Abstract Objective Artificial intelligence (AI) models may propagate harmful biases in performance and hence negatively affect the underserved. We aimed to assess the degree to which data quality of electronic health records (EHRs) affected by inequities related to low socioeconomic status (SES), results in differential performance of AI models across SES. Materials and Methods This study utilized existing machine learning models for predicting asthma exacerbation in children with asthma. We compared balanced error rate (BER) against different SES levels measured by HOUsing-based SocioEconomic Status measure (HOUSES) index. As a possible mechanism for differential performance, we also compared incompleteness of EHR information relevant to asthma care by SES. Results Asthmatic children with lower SES had larger BER than those with higher SES (eg, ratio = 1.35 for HOUSES Q1 vs Q2–Q4) and had a higher proportion of missing information relevant to asthma care (eg, 41% vs 24% for missing asthma severity and 12% vs 9.8% for undiagnosed asthma despite meeting asthma criteria). Discussion Our study suggests that lower SES is associated with worse predictive model performance. It also highlights the potential role of incomplete EHR data in this differential performance and suggests a way to mitigate this bias. Conclusion The HOUSES index allows AI researchers to assess bias in predictive model performance by SES. Although our case study was based on a small sample size and a single-site study, the study results highlight a potential strategy for identifying bias by using an innovative SES measure.