Dhruva, Sanket S.

Abstract Acute myocardial infarction complicated by cardiogenic shock (AMI-CS) is a heterogeneous clinical syndrome associated with substantial morbidity and mortality. We developed a machine learning-based mortality model to identify features and patient subgroups associated with the largest change in mortality risk when evaluating treatment with Impella devices or intra-aortic Balloon Pump (IABP). Our cohort comprised 369 sites and 15,796 patient visits to the cardiac catheterization laboratory from the National Cardiovascular Data Registry. The estimated population mean excess mortality effect of treatment with Impella devices vs IABP was 10.4 ± 0.8%. However, we identified clinical subgroups of 282 patients for whom a decreased risk of mortality was associated with use of Impella as compared with IABP. Those patients were on average younger, presented with higher systolic blood pressure, higher rate of salvage percutaneous coronary intervention, higher initial creatinine, and lower hemoglobin. While Impella devices were associated with higher mortality risk overall, certain clinical profiles were associated with lower risk, illustrating heterogeneity of treatment effects.

This cross-sectional study assesses published research articles about medical items and services approved by the Coverage With Evidence Development program to characterize their scientific impact.

Introduction In 2005, the Centers for Medicare and Medicaid Services introduced the Coverage with Evidence Development program for items and services with limited evidence of benefit and harm for Medicare beneficiaries, aiming to generate evidence to determine whether they meet the statutory “reasonable and necessary” criteria for coverage. Coverage with Evidence Development requires participation in clinical studies approved by the Centers for Medicare and Medicaid Services (i.e. Coverage with Evidence Development-approved studies) as a condition of coverage. We examined the quality of evidence generated by Coverage with Evidence Development-approved studies compared with those that informed Centers for Medicare and Medicaid Services’ initial Coverage with Evidence Development decisions (i.e. National Coverage Determination studies). Methods Using Centers for Medicare and Medicaid Services’ webpage, we identified all items and services covered under Coverage with Evidence Development and their Coverage with Evidence Development-approved studies. Through searching PubMed and Google Scholar, we identified original research articles that reported results for primary endpoints of Coverage with Evidence Development-approved studies. We then reviewed the initial Coverage with Evidence Development decision memos and identified National Coverage Determination studies that were original research.We characterized and compared Coverage with Evidence Development-approved studies and National Coverage Determination studies. Results From 2005 to 2023, 26 items and services were covered under the Coverage with Evidence Development program, associated with 196 National Coverage Determination studies (170 (86.7%) clinical trials and 26 (13.3%) registries) and 116 unique Coverage with Evidence Development-approved studies (86 (74.1%) clinical trials, 23 (19.8%) registries, 4 (3.4%) claims-based studies, and 3 (2.6%) expanded access studies). Among clinical trial studies, National Coverage Determination studies and Coverage with Evidence Development-approved studies did not differ with respect to multi-arm design (59.4% vs 68.6%; p = 0.15). However, among multi-arm clinical trial studies, National Coverage Determination studies were less likely than Coverage with Evidence Development-approved studies to be randomized (52.5% vs 93.2%; p < 0.001). Overall, National Coverage Determination studies less frequently had ≥ 1 primary endpoint focused on a clinical outcome measure (65.8% vs 87.9%; p = 0.006) and less frequently exclusively enrolled Medicare beneficiaries (3.1% vs 25.9%; p < 0.001). In addition, National Coverage Determination studies had smaller population sizes than Coverage with Evidence Development-approved studies (median 100 (interquartile range, 45–414) vs 302 (interquartile range, 93–1000) patients; p = 0.002). Among Coverage with Evidence Development-approved studies, 59 (50.9%) had not yet publicly reported results for the primary endpoint. Discussion Studies required under Medicare’s Coverage with Evidence Development program more often used randomized study designs, had larger patient populations, enrolled US patients, and focused on clinical outcomes as primary endpoints than studies used to inform initial National Coverage Determinations. However, not all Coverage with Evidence Development-approved studies have thus far reported results, which could create uncertainty for patients, physicians, and payers regarding the clinical benefits of covered items and services. Conclusion The Centers for Medicare and Medicaid Services’ Coverage with Evidence Development program has successfully prompted generation of more robustly designed clinical studies to better inform clinical, regulatory, and coverage decisions compared to studies informing initial coverage decisions. Yet, opportunities exist to further strengthen the design and dissemination of studies required under this program.

Abstract The Centers for Medicare & Medicaid Services (CMS) relies on public comments submitted in response to proposed national coverage determinations to assist the agency in determining the coverage of items and services for Medicare beneficiaries. In a cross-sectional study, we characterized the cited evidence and what funding supported the cited evidence submitted in public comments to CMS for all therapeutic medical device national coverage determinations finalized between June 2019 and June 2022. Of 681 public comments, 159 (23%) cited at least 1 identifiable published scientific journal article. Within these 159 public comments, 198 unique articles were cited, 170 (86%) of which included funding statements or author disclosures. Among these, 96 (56%) disclosed funding from manufacturers that would benefit from Medicare coverage and/or were written by author(s) who received funding from these manufacturers. In summary, most public commenters for national coverage determinations did not cite published scientific journal articles to support their positions. Among those who did, more than half of articles were directly funded by manufacturers that would benefit from coverage. Greater funding of independent, non–industry-supported research may help provide unbiased evaluations of benefits and harms to support Medicare coverage decisions.

Observational medical data present unique opportunities for analysis of medical outcomes and treatment decision making. However, because these datasets do not contain the strict pairing of randomized control trials, matching techniques are to draw comparisons among patients. A key limitation to such techniques is verification that the variables used to model treatment decision making are also relevant in identifying the risk of major adverse events. This article explores a deep mixture of experts approach to jointly learn how to match patients and model the risk of major adverse events in patients. Although trained with information regarding treatment and outcomes, after training, the proposed model is decomposable into a network that clusters patients into phenotypes from information available before treatment. This model is validated on a dataset of patients with acute myocardial infarction complicated by cardiogenic shock. The mixture of experts approach can predict the outcome of mortality with an area under the receiver operating characteristic curve of 0.85 ± 0.01 while jointly discovering five potential phenotypes of interest. The technique and interpretation allow for identifying clinically relevant phenotypes that may be used both for outcomes modeling as well as potentially evaluating individualized treatment effects.

Background/aims There has been growing interest in better understanding the potential of observational research methods in medical product evaluation and regulatory decision-making. Previously, we used linked claims and electronic health record data to emulate two ongoing randomized controlled trials, characterizing the populations and results of each randomized controlled trial prior to publication of its results. Here, our objective was to compare the populations and results from the emulated trials with those of the now-published randomized controlled trials. Methods This study compared participants’ demographic and clinical characteristics and study results between the emulated trials, which used structured data from OptumLabs Data Warehouse, and the published PRONOUNCE and GRADE trials. First, we examined the feasibility of implementing the baseline participant characteristics included in the published PRONOUNCE and GRADE trials’ using real-world data and classified each variable as ascertainable, partially ascertainable, or not ascertainable. Second, we compared the emulated trials and published randomized controlled trials for baseline patient characteristics (concordance determined using standardized mean differences <0.20) and results of the primary and secondary endpoints (concordance determined by direction of effect estimates and statistical significance). Results The PRONOUNCE trial enrolled 544 participants, and the emulated trial included 2226 propensity score-matched participants. In the PRONOUNCE trial publication, one of the 32 baseline participant characteristics was listed as an exclusion criterion on ClinicalTrials.gov but was ultimately not used. Among the remaining 31 characteristics, 9 (29.0%) were ascertainable, 11 (35.5%) were partially ascertainable, and 10 (32.2%) were not ascertainable using structured data from OptumLabs. For one additional variable, the PRONOUNCE trial did not provide sufficient detail to allow its ascertainment. Of the nine variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 6 (66.7%). The primary endpoint of time from randomization to the first major adverse cardiovascular event and secondary endpoints of nonfatal myocardial infarction and stroke were concordant between the emulated trial and published randomized controlled trial. The GRADE trial enrolled 5047 participants, and the emulated trial included 7540 participants. In the GRADE trial publication, 8 of 34 (23.5%) baseline participant characteristics were ascertainable, 14 (41.2%) were partially ascertainable, and 11 (32.4%) were not ascertainable using structured data from OptumLabs. For one variable, the GRADE trial did not provide sufficient detail to allow for ascertainment. Of the eight variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 4 (50.0%). The primary endpoint of time to hemoglobin A1c ≥7.0% was mostly concordant between the emulated trial and the published randomized controlled trial. Conclusion Despite challenges, observational methods and real-world data can be leveraged in certain important situations for a more timely evaluation of drug effectiveness and safety in more diverse and representative patient populations.

Abstract The objective of this study is to describe application of the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) to support medical device real-world evaluation in a National Evaluation System for health Technology Coordinating Center (NESTcc) Test-Case involving 2 healthcare systems, Mercy Health and Mayo Clinic. CDM implementation was coordinated across 2 healthcare systems with multiple hospitals to aggregate both medical device data from supply chain databases and patient outcomes and covariates from electronic health record data. Several data quality assurance (QA) analyses were implemented on the OMOP CDM to validate the data extraction, transformation, and load (ETL) process. OMOP CDM-based data of relevant patient encounters were successfully established to support studies for FDA regulatory submissions. QA analyses verified that the data transformation was robust between data sources and OMOP CDM. Our efforts provided useful insights in real-world data integration using OMOP CDM for medical device evaluation coordinated across multiple healthcare systems.

Abstract Objective To emulate the GRADE (Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness Study) trial using real world data before its publication. GRADE directly compared second line glucose lowering drugs for their ability to lower glycated hemoglobin A 1c (HbA 1c ). Design Observational study. Setting OptumLabs® Data Warehouse (OLDW), a nationwide claims database in the US, 25 January 2010 to 30 June 2019. Participants Adults with type 2 diabetes and HbA 1c 6.8-8.5% while using metformin monotherapy, identified according to the GRADE trial specifications, who also used glimepiride, liraglutide, sitagliptin, or insulin glargine. Main outcome measures The primary outcome was time to HbA 1c ≥7.0%. Secondary outcomes were time to HbA 1c >7.5%, incident microvascular complications, incident macrovascular complications, adverse events, all cause hospital admissions, and all cause mortality. Propensity scores were estimated using the gradient boosting machine method, and inverse propensity score weighting was used to emulate randomization of the treatment groups, which were then compared using Cox proportional hazards regression. Results 8252 people were identified (19.7% of adults starting the study drugs in OLDW) who met eligibility criteria for the GRADE trial (glimepiride arm=4318, liraglutide arm=690, sitagliptin arm=2993, glargine arm=251). The glargine arm was excluded from analyses owing to small sample size. Median times to HbA 1c ≥7.0% were 442 days (95% confidence interval 394 to 480 days) for glimepiride, 764 (741 to not calculable) days for liraglutide, and 427 (380 to 483) days for sitagliptin. Liraglutide was associated with lower risk of reaching HbA 1c ≥7.0% compared with glimepiride (hazard ratio 0.57, 95% confidence interval 0.43 to 0.75) and sitagliptin (0.55, 0.41 to 0.73). Results were consistent for the secondary outcome of time to HbA 1c >7.5%. No significant differences were observed among treatment groups for the remaining secondary outcomes. Conclusions In this emulation of the GRADE trial, liraglutide was statistically significantly more effective at maintaining glycemic control than glimepiride or sitagliptin when added to metformin monotherapy. Generating timely evidence on medical treatments using real world data as a complement to prospective trials is of value.