Abstract
Background: Patients’ choices of providers when undergoing elective surgeries significantly impact both perioperative outcomes and costs. There exist a variety of approaches that are available to patients for evaluating between different hospital choices.
Objective: This paper aims to compare differences in outcomes and costs between hospitals ranked using popular internet-based consumer ratings, quality stars, reputation rankings, average volumes, average outcomes, and precision machine learning–based rankings for hospital settings performing hip replacements in a large metropolitan area.
Methods: Retrospective data from 4192 hip replacement surgeries among Medicare beneficiaries in 2018 in the Chicago metropolitan area were analyzed for variations in outcomes (90-day postprocedure hospitalizations and emergency department visits) and costs (90-day total cost of care) between hospitals ranked through multiple approaches.
Results: Only a minority of patients (27.6% to 49.6%) were found to be matched to higher-ranked hospitals for each of the different approaches. Of the approaches considered, hip replacements at hospitals that were more highly ranked by consumer ratings, quality stars, and machine learning were all consistently associated with improvements in outcomes and costs. The improvement was greatest across all metrics and analyses for machine learning–based rankings.
Conclusions: There may be a substantive opportunity to increase the number of patients matched to appropriate hospitals across a broad variety of ranking approaches. Elective hip replacement surgeries performed at hospitals where patients were matched based on patient-specific machine learning were associated with better outcomes and lower total costs of care.
Introduction
When a patient needs elective surgery, such as a hip replacement, they are often overwhelmed by choice. To decide where to go, they might look at "best hospital" lists from US News, check Yelp reviews, or look at government-issued CMS star ratings.
The fundamental problem is that these traditional rankings are static and global. They treat hospital quality as a "one-size-fits-all" metric, assuming that if a hospital is "the best," it is the best for every patient. However, a hospital that excels at treating healthy 60-year-olds might have higher complication rates for 80-year-olds with multiple comorbidities. This study demonstrates that a personalized machine learning approach—"Precision Navigation"—significantly outperforms traditional consumer and quality metrics by matching specific patients to hospitals based on their unique health profiles.
Methods and Data
The researchers conducted a retrospective study using Medicare Fee-For-Service (FFS) claims data from 2013 to 2018. They focused on 4,192 hip replacement surgeries performed across 69 hospitals in the Chicago metropolitan area.
Ranking Methodologies Compared
- Consumer/Reputation: Yelp, Healthgrades, and US News.
- Government Quality: CMS Hospital Compare (Star ratings).
- Volume/Outcome Metrics: Average yearly surgical volume and average historical outcome rates.
- Precision Navigation: A machine learning (ML) system (developed by Health at Scale Corp) that uses a patient’s personalized medical characteristics to predict which hospital is likely to yield the best long-term outcome for that specific person.
Analytical Framework
Data from 2013–2017 was used to train the Precision Navigation ML model, while data from 2018 served as the evaluation set. To ensure a fair comparison, the authors used propensity-based adjusted analysis. This statistical technique matches patients with similar health risks (using the Elixhauser comorbidity score) to ensure that the ML model wasn't simply "cherry-picking" healthier patients to improve its results.
Key Insights
The "Gray Zone" of Quality
The authors found that over 85% of the hospitals in the study fell into a "gray zone"—they weren't universally good or bad, but rather selectively good or bad for different types of patients. This highlights why global rankings often fail to provide accurate guidance for individual risk profiles.
Load Balancing vs. Overcrowding
Traditional rankings tend to funnel everyone toward a tiny handful of "top" hospitals, which can lead to overcrowding and inefficiency. In contrast, the ML approach identified 54 different hospitals as "top-ranked" for at least one patient. This suggests that personalized matching could better distribute the surgical workload across a region, utilizing the specific strengths of more facilities.
The Opportunity Gap
Only about 27.6% of patients in the study were actually treated at the hospital the ML model would have recommended as their "top" choice. This suggests a massive opportunity to improve public health by better steering patients toward hospitals optimized for their specific needs.
Results
The personalized ML approach (Precision Navigation) was the clear winner across all metrics in both adjusted and unadjusted analyses:
- Clinical Outcomes: Compared to the population average, patients matched via the ML approach saw a 3.4% reduction in 90-day hospitalizations and a 4.1% reduction in Emergency Department (ED) visits.
- Cost Savings: The ML approach was associated with a $3,315 reduction in the 90-day Total Cost of Care (TCOC) per patient.
- Consistency: While CMS stars and Yelp showed some correlation with better outcomes, the ML approach was the only method that consistently showed significant improvements across every metric.
Discussion and Future Work
While the results are compelling, the study relied on administrative claims data, which lacks the deep clinical nuance found in Electronic Health Records (EHRs). Furthermore, the study was limited to the Chicago area and a Medicare population; further research is needed to see if these results hold true for younger, privately insured patients or in different geographic regions.
As a retrospective study, it shows a strong association but cannot definitively prove causality. However, the findings suggest a substantive opportunity to increase the number of patients matched to appropriate hospitals. Future directions include expanding this personalized matching framework to other high-volume procedures and integrating it into the actual referral workflows used by doctors and insurance providers.
Summary updated