Citations

Abstract Objective In 2024, the US News and World Report (USNWR)'s Best Hospitals began reporting complications following elective outpatient otolaryngology procedures in its ears, nose, and throat (ENT) ranking. However, this methodology has not been clinically adjudicated in a peer‐reviewed domain. We aimed to conduct manual health record adjudication to assess whether the USNWR methodology accurately attributes complications following elective outpatient otolaryngology procedures. Study Design Case‐cohort. Setting A case‐cohort of n = 80 elective, outpatient otolaryngology procedure/complication pairings from January 2019 to October 2023 across four academic medical centers using the 2024 USNWR otolaryngology procedure subgroups and Solventum Ambulatory Potentially Preventable Complications (AM‐PPC) software. Methods We used the AM‐PPC‐defined complication types as our standard of comparison for our clinical adjudication. Using the electronic health record, we adjudicated: (1) Was there agreement (yes/no) on complication type between our clinical adjudication and the Solventum AM‐PPC software? (2) Was the complication clinically related to, or a sequela of, the index otolaryngology procedure (yes/no)? Results The clinical adjudication agreed on complication type in 59/80 (74%) of patients, and 63/80 (79%) were judged to be clinically related to the index otolaryngology procedure. In total, 50/80 (63%) complications were adjudicated to be in agreement on both the complication type and clinical relatedness to the index procedure, while 9/80 (11%) complications did not agree with the complication type nor were clinically related to the index procedure. Conclusion Outpatient complications reported in the USNWR ENT rating lack the specificity to exclude patients whose complications are sequelae of chronic underlying conditions or related to concurrent, non‐otolaryngology procedures.

BACKGROUND: Publicly reported 30-day acute myocardial infarction (AMI) mortality rates are widely used to benchmark hospital performance. However, these measures may lack validity when they include deaths among patients with do-not-resuscitate (DNR) orders or those receiving palliative care—groups for whom mortality may be expected and goal-concordant. METHODS: We analyzed the national 100% Medicare Inpatient Standard Analytic File and Medicare Beneficiary Summary file to assess 30-day AMI mortality rates stratified by DNR status (absent, present-on-admission, and postadmission) and palliative care involvement. We then conducted a retrospective chart review of all 30-day AMI mortalities at a single academic medical center (2019–2025), applying Global Registry of Acute Coronary Events and a Palliative Performance Scale scores to identify unpreventable deaths, which we defined as a Global Registry of Acute Coronary Events >190.5 or Palliative Performance Scale ≤40%. Global Registry of Acute Coronary Events is a risk-stratification tool using clinical and laboratory variables whereas the Palliative Performance Scale is a bedside functional status and palliative care assessment quantifying patient’s self-care ability. RESULTS: Across n=467 259 total AMI encounters, mean (SD) age was 75.1 (10.5), and the population was 44% female. Thirty-day mortality was 10.4% (39 112/376 062) among patients without DNRs, 43.1% (26 181/60 719) in those with present-on-admission DNR, and 72.1% (21 984/30 478) in those with postadmission DNR. Among 9.2% (43 090/467 259) of encounters with palliative care, there was a 76.9% mortality rate (33 151/43 090). Hazard ratios for mortality were 5.25 (95% CI, 5.17–5.33) for present-on-admission DNR and 10.73 (95% CI, 10.55–10.91) for postadmission DNR as compared with encounters without DNRs. Among n=24 AMI mortalities at our institution, 71% (17/24) had DNRs, and 33% (8/24) met the unpreventable death criteria. CONCLUSIONS: The validity of 30-day AMI mortality metrics may be undermined by their inclusion of high-acuity, end-of-life patients, which may disincentivize hospitals from delivering appropriate, patient-centered care. Removal of DNR or palliative cases from performance measures should be explored to better identify opportunities for reducing preventable deaths.

ImportanceHospital ratings including the US News & World Report’s Best Hospitals rankings and the Centers for Medicare & Medicaid Services’ (CMS’) Overall Hospital Quality Star Rating (Overall Star Rating) measure different outcomes and are weakly correlated. Therefore, methods for defining and measuring reliable excellence, defined as consistently great performance across all quality measures, are needed.ObjectiveTo assess a measure of reliable excellence using the 45 quality measures reported in the Overall Star Ratings.Design, Setting, and ParticipantsThis cross-sectional study used hospital-level data from the 2023 and 2024 CMS Overall Star Ratings at all US hospitals with a 2023 and 2024 CMS Overall Star Rating.ExposuresThe exposure was the CMS Overall Star Rating summary score, a continuous variable calculated from the weighted z scores of 45 quality measures used in the Overall Star Rating. A total of 100 000 simulations were run in which all US hospitals’ CMS Overall Star Rating summary scores were calculated through summation of z scores from randomly generated measure weights, as opposed to the existing weights used in the Overall Star Rating method.Main Outcomes and MeasuresReliable excellence, defined as achieving a 90th percentile (or better) CMS Overall Star Rating summary score on at least 50 000 of 100 000 simulations. The percentage of hospitals achieving reliable excellence was calculated both overall and stratified by CMS Ovearll Star Ratings.ResultsThere were 2700 hospitals in the analysis, with 335 5-star hospitals (12.4%), 727 4-star hospitals (26.9%), 799 3-star hospitals (29.6%), 572 2-star hospitals (21.2%), and 267 1-star hospitals (9.9%) in the 2024 CMS Overall Star Rating. A total of 244 of 2700 hospitals (9.0%) met the study definition of reliable excellence, whereas 1287 of 2700 hospitals (47.7%) achieved excellence in at least 1 simulation.Conclusions and RelevanceThis cross-sectional study of 2700 US hospitals found that only 244 hospitals (9.0%), including less than two-thirds of the CMS 5-star rated hospitals, were reliably excellent across 100 000 CMS Overall Star Rating scoring simulations using random measure weightings. These findings lend credence to the ubiquity of inconsistent greatness in health care quality and illuminate the need for methods to distinguish hospitals that provide reliably excellent care.

AbstractNational performance metrics ultimately enhance patient decision‐making and promote meaningful improvements in health care delivery, which makes having valid and reliable measures essential. This study examined US News and World Report metrics from 2019 to 2012 and used electronic health record data, combined with detailed chart review across 3 in‐system hospitals, to assess the provision of care compared to the attribution of patients assigned to the ear, nose, and throat (ENT) mortality group. Of the initial 47 ENT‐attributed deaths, 23 of those were verified, dimensioning the mortality rate from 1.7% to just 0.8%. These results underscore the necessity of rethinking measures and mortality attribution methodologies to be more accurate. Current methods use Medicare Severity Diagnosis Related Group billing coding to map the attribution. We suggest transitioning away from specialty ranking approaches and towards a procedure and condition “rating” approach to ensure that these ranking types capture data about the provision of care within a given encounter.

Background The COVID-19 pandemic introduced unique challenges to healthcare systems, particularly in relation to patient safety and adverse events during hospitalization. There is limited understanding of COVID-19’s association with some patient safety indicators (PSIs). Objectives This study aimed to investigate the impact of COVID-19 infection on the rate of PSI-3 events and its implications on quality metrics. We compared PSI-3 event rates between COVID-19–infected and uninfected patients and examined the clinical characteristics of COVID-19 patients experiencing PSI-3 events. Methods This is a retrospective study at Mayo Clinic hospitals between January 2020 and February 2022, analyzing patients meeting PSI-3 denominator eligibility criteria. PSI-3 events were identified using AHRQ WinQI software. Patients were categorized based on COVID-19 status. Patient demographics, characteristics, and PSI-3 rates were compared. A case series analysis described clinical details of COVID-19 patients with PSI-3 events. Results Of 126,781 encounters meeting PSI-3 criteria, 8674 (6.8%) had acute COVID-19 infection. COVID-19–infected patients were older, more likely to be male, non-white, and had private insurance. PSI-3 rates were significantly higher in COVID-19 patients (0.21% versus 0.06%, P < 0.0001), even after risk adjustment (adjusted risk ratio, 3.24, P < 0.0001). The case series of 17 COVID-19 patients with PSI-3 events showed distinctive clinical characteristics, including higher medical device–related pressure injuries, and greater predisposition for head, face, and neck region. Conclusions Acute COVID-19 infection correlates with higher PSI-3 event rates. Current quality indicators may require adaptation to address the pandemic’s complexities and impact on patient safety. Further research is needed to comprehensively understand the intricate relationship between COVID-19 and patient outcomes.

AbstractPurposeBy assessing longitudinal associations between COVID‐19 census burdens and hospital characteristics, such as bed size and critical access status, we can explore whether pandemic‐era hospital quality benchmarking requires risk‐adjustment or stratification for hospital‐level characteristics.MethodsWe used hospital‐level data from the US Department of Health and Human Services including weekly total hospital and COVID‐19 censuses from August 2020 to August 2023 and the 2021 American Hospital Association survey. We calculated weekly percentages of total adult hospital beds containing COVID‐19 patients. We then calculated the number of weeks each hospital spent at Extreme (≥20% of beds occupied by COVID‐19 patients), High (10%–19%), Moderate (5%–9%), and Low (<5%) COVID‐19 stress. We assessed longitudinal hospital‐level COVID‐19 stress, stratified by 15 hospital characteristics including joint commission accreditation, bed size, teaching status, critical access hospital status, and core‐based statistical area (CBSA) rurality.FindingsAmong n = 2582 US hospitals, the median(IQR) weekly percentage of hospital capacity occupied by COVID‐19 patients was 6.7%(3.6%–13.0%). 80,268/213,383 (38%) hospital‐weeks experienced Low COVID‐19 census stress, 28% Moderate stress, 22% High stress, and 12% Extreme stress. COVID‐19 census burdens were similar across most hospital characteristics, but were significantly greater for critical access hospitals.ConclusionsUS hospitals experienced similar COVID‐19 census burdens across multiple institutional characteristics. Evidence‐based inclusion of pandemic‐era outcomes in hospital quality reporting may not require significant hospital‐level risk‐adjustment or stratification, with the exception of rural or critical access hospitals, which experienced differentially greater COVID‐19 census burdens and may merit hospital‐level risk‐adjustment considerations.