Abstract
We experiment with recent ensemble machine learning methods in estimating healthcare costs, utilizing Finnish data containing rich individual-level information on healthcare costs, socioeconomic status and diagnostic data from multiple registries. Our data are a random 10% sample (553,675 observations) from the Finnish population in 2017. Using annual healthcare cost in 2017 as a response variable, we compare the performance of Random forest, Gradient Boosting Machine (GBM) and eXtreme Gradient Boosting (XGBoost) to linear regression. As machine learning methods are often seen as unsuitable in risk adjustment applications because of their relative opaqueness, we also introduce visualizations from the machine learning literature to help interpret the contribution of individual variables to the prediction. Our results show that ensemble machine learning methods can improve predictive performance, with all of them significantly outperforming linear regression, and that a certain level of interpretation can be provided for them. We also find individual-level socioeconomic variables to improve prediction accuracy and that their effect is larger for machine learning methods. However, we find that the predictions used for funding allocations are sensitive to model selection, highlighting the need for comprehensive robustness testing when estimating risk adjustment models used in applications.
Introduction
In public healthcare systems, such as those in Finland or the UK, governments must decide how to distribute billions of euros to regional authorities. To do this fairly, they use "risk adjustment" models: formulas that predict how much a region should cost based on the "need" of its population, accounting for factors like age, chronic diseases, and socioeconomic status.
The industry standard is Ordinary Least Squares (OLS) regression because it is transparent and easy to explain. However, OLS struggles to capture complex, non-linear interactions between health and social factors. While advanced machine learning (ML) could offer better accuracy, it is often avoided because it is seen as a "black box." This paper investigates whether the accuracy gains of ML are worth the loss of transparency and how we might "peek inside" these models to make them palatable for policy.
Methodology and Data
The researchers compared four models: Linear Regression (OLS), Random Forest (RF), Gradient Boosting Machine (GBM), and XGBoost. To address the "black box" concern, they applied three interpretability techniques: Variable Importance ranking, Partial Dependence Plots (PDP), and LIME (Local Interpretable Model-agnostic Explanations).
The study utilized a massive dataset from Finnish national registries, consisting of a random 10% sample of the population in 2017 (553,675 observations). Features included demographics, 51 clinical disease categories, and rich socioeconomic data like household income and education. The target variable was total annual public healthcare expenditure per person.
Key Insights and Results
The Socioeconomic Synergy
One of the most striking findings was that socioeconomic variables (like income) improved the performance of ML models much more than they improved the OLS model. This suggests that ML is successfully capturing complex interactions—for example, how poverty might exacerbate the costs of a specific chronic illness—that linear models simply miss.
The Performance Ceiling and Skewness
By using flexible ML models, the authors established a "performance ceiling" for the data. They found that even the best models only explain about 17% of the variance in costs. This "low" $R^2$ is typical in healthcare, where costs are highly skewed by rare, expensive events (as shown in the figure above), but ML still offered a 25% relative improvement over OLS. Random Forest achieved the highest $R^2$ (0.1702) compared to OLS (0.1368).
The Robustness Trap
Even when two models have similar overall accuracy, they might disagree on how much money a specific region should get. For the region of Ostrobothnia, the difference between models amounted to tens of millions of euros. This "model sensitivity" is a major hurdle for political legitimacy. Furthermore, while cancer and age were top predictors for boosting models, Random Forest placed a much higher emphasis on household income, ranking it as the most important variable.
Conclusion and Policy Implications
Interpretability tools like PDPs showed that Random Forest's predictions for the elderly aligned better with clinical intuition than boosting models. However, the LIME technique showed low "explanation fit," suggesting that explaining specific individual scores remains difficult.
The authors conclude that while ML is ready for benchmarking provider performance, using it for actual funding formulas requires more rigorous robustness testing to ensure that algorithmic changes do not unfairly impact regional budgets. Future work should explore more comprehensive diagnostic feature sets to see if model predictions converge more closely.
Summary updated