Citations

Moritz F Sinner, 2012 Finalist and Presenting Author

Importance Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed. Observations While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings. Conclusion and Relevance The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

We are at a pivotal moment in the development of healthcare artificial intelligence (AI), a point at which enthusiasm for machine learning has not caught up with the scientific evidence to support the equity and accuracy of diagnostic and therapeutic algorithms. This proposal examines algorithmic biases, including those related to race, gender and socioeconomic status, and accuracy, including the paucity of prospective studies and lack of multisite validation. We then suggest solutions to these problems. We describe the Mayo Clinic, Duke University, Change Healthcare project that is evaluating 35.1 billion healthcare records for bias. And we propose ‘Ingredients’ style labels and an AI evaluation/testing system to help clinicians judge the merits of products and services that include algorithms. Said testing would include input data sources and types, dataset population composition, algorithm validation techniques, bias assessment evaluation and performance metrics.

Background: The 2013 ACC/AHA Cholesterol Guidelines departed from prior ATPIII guidelines, recommending statin treatment based on patients’ predicted 10-year ASCVD risk and de-emphasizing LDL-C targets. However, the degree to which these new guidelines have been adopted by clinicians in community practice is unknown. Methods: We surveyed 522 clinicians in 133 cardiology, primary care, and endocrinology practices participating in the Patient and Provider Assessment of Lipid Management (PALM) Registry. Clinicians were asked about their likelihood of prescribing statins in 4 hypothetical patient scenarios. Results: Among responders, 79% were physicians (60% cardiologists, 38% primary care or general practice, 2% endocrinology), 18% were advanced practice providers, and 3% trainees. When queried, 73% of providers reported primarily using the 2013 ACC/AHA Guidelines to guide lipid management. However, only half reported “often” or “always” calculating 10-year ASCVD risk to aid lipid management decisions for primary prevention. When presented with a high 10-year ASCVD risk patient (>15%) with a high LDL-C (150 mg/dL), 92% of providers recommended statins. Similarly, 83% of providers would use statins in a diabetic patient with a LDL-C of 140 mg/dL. Yet, only 37% reported they would prescribe a statin to an older adult (age 70) for whom guidelines recommend statin based on high 10-year ASCVD risk alone (10.1%). And up to 40% would prescribe a statin to a younger (age 40) with a low 10-year risk, but a high LDL (165 mg/dL) who would not otherwise meet criteria for statin therapy under the new guidelines. Cardiologists and non-cardiologists were similarly likely prescribe statins in most scenarios with the exception of to the younger patient with elevated LDL-C were cardiologists used statins more commonly (32% vs. 42%. OR 1.5, 95% CI 1.1-2.2, p=0.02). Conclusion: Initial survey results show that while most clinicians claim adoption of the 2013 ACC/AHA Guidelines, reported practices are not always adherent to these guidelines. Providers do not always calculate 10-year ASCVD risk to guide lipid management and many do not prescribe statins to patients eligible based on 10-year ASCVD risk.

Background Tools for the prediction of atrial fibrillation ( AF ) may identify high‐risk individuals more likely to benefit from preventive interventions and serve as a benchmark to test novel putative risk factors. Methods and Results Individual‐level data from 3 large cohorts in the U nited S tates (Atherosclerosis Risk in Communities [ ARIC ] study, the Cardiovascular Health Study [ CHS ], and the Framingham Heart Study [ FHS ]), including 18 556 men and women aged 46 to 94 years (19% A frican A mericans, 81% whites) were pooled to derive predictive models for AF using clinical variables. Validation of the derived models was performed in 7672 participants from the Age, Gene and Environment—Reykjavik study ( AGES ) and the Rotterdam Study ( RS ). The analysis included 1186 incident AF cases in the derivation cohorts and 585 in the validation cohorts. A simple 5‐year predictive model including the variables age, race, height, weight, systolic and diastolic blood pressure, current smoking, use of antihypertensive medication, diabetes, and history of myocardial infarction and heart failure had good discrimination (C‐statistic, 0.765; 95% CI , 0.748 to 0.781). Addition of variables from the electrocardiogram did not improve the overall model discrimination (C‐statistic, 0.767; 95% CI , 0.750 to 0.783; categorical net reclassification improvement, −0.0032; 95% CI , −0.0178 to 0.0113). In the validation cohorts, discrimination was acceptable ( AGES C‐statistic, 0.664; 95% CI , 0.632 to 0.697 and RS C‐statistic, 0.705; 95% CI , 0.664 to 0.747) and calibration was adequate. Conclusion A risk model including variables readily available in primary care settings adequately predicted AF in diverse populations from the U nited S tates and E urope.

Background— Data regarding the familial aggregation of left ventricular (LV) geometry and its relations to parental heart failure (HF) are limited. Methods and Results— We evaluated concordance of LV geometry within 1093 nuclear families in 5758 participants of the original (parents) (n=2351) and offspring (n=3407) cohorts of the Framingham Heart Study undergoing routine echocardiography in mid- to late adulthood. LV geometry was categorized based on cohort- and sex-specific 80th percentile cutoffs of LV mass and relative wall thickness (RWT) into normal (both <80th percentile), concentric remodeling (LV mass <80th percentile; RWT >80th percentile), concentric hypertrophy (both >80th percentile), and eccentric hypertrophy (LV mass >80th percentile; RWT <80th percentile). Within nuclear families, LV geometry was concordant among related pairs (parent-child, sibling-sibling) ( P =0.0015) but not among unrelated spousal pairs ( P =0.60), a finding that remained unchanged after adjusting for clinical covariates known to influence LV remodeling (age, systolic blood pressure, body mass index), excluding individuals with prevalent HF and myocardial infarction, and varying the thresholds for defining LV geometry. The prevalence of abnormal LV geometry was higher in family members of affected individuals, with recurrence risks of 1.4 for concentric remodeling (95% CI, 1.2 to 1.7) and eccentric hypertrophy (95% CI, 1.1 to 1.8) and 3.9 (95% CI, 3.2 to 4.6) for concentric hypertrophy. In a subset of 1497 offspring, we observed an association between parental HF (n=458) and eccentric hypertrophy in offspring ( P <0.0001). Conclusions— Our investigation of a 2-generational community-based sample demonstrates familial aggregation of LV geometry, with the greatest recurrence risk for concentric LV geometry, and establishes an association between eccentric LV geometry and parental HF.