TL;DR
Researchers in Aotearoa New Zealand have demonstrated that machine learning (ML) can accurately predict cardiovascular disease (CVD) risk in diabetic populations using only "social administrative data"—national records like census info and hospitalizations—rather than traditional clinical markers. Crucially, the study found that high-performing models like Xgboost do not require a trade-off between overall predictive accuracy and ethnic equity, suggesting that population-level health planning can be both precise and fair.
What Problem Does This Paper Address?
Cardiovascular disease is the leading cause of death globally, and those living with diabetes face a significantly higher risk. In New Zealand, this burden is not shared equally; Māori, Pasifika, and Asian populations face major health inequities driven by socio-economic and systemic factors.
While many CVD risk prediction models exist (like the NZ PREDICT equations), they typically rely on "rich" clinical data—think BMI, blood pressure, and specific biomarkers. These are great for a doctor's office but difficult to deploy at a national scale for population health planning. The authors sought to determine if "administrative data" (records already collected by the government for non-clinical purposes) could be used to predict complications and whether these models would remain fair across different ethnic groups.
Methods
The researchers compared five different ML architectures:
- Logistic Regression (The baseline)
- Decision Trees
- Random Forest (RF)
- Neural Networks
- Xgboost (Extreme Gradient Boosting)
What makes their approach novel is the Optimization Scenarios. They didn't just train one model; they tested three strategies to see how to best handle ethnicity:
- Scenario A: Train and optimize on the whole population.
- Scenario B: Train on the whole population but optimize the hyperparameters specifically for a single ethnic group.
- Scenario C: Train and optimize only on data from a specific ethnic group.
They used an 80/20 train/test split and validated the models against a future time period (2016–2018) to ensure the predictions held up over time.
Datasets
The study utilized the Stats NZ Integrated Data Infrastructure (IDI), a massive, linked research database. Specific datasets included:
- Census 2013: For smoking status, language, and employment.
- Ministry of Health Chronic Condition Table: To identify individuals with diabetes and other comorbidities.
- Pharmaceutical Data: To track preventive medications.
- IDI Population Explorer: For social indicators like social housing use and major life events (e.g., divorce).
The final cohort included approximately 146,000 New Zealand residents aged 30–74 with known diabetes but no prior history of CVD.
Key Insights and Ideas
- Administrative Data is a Viable Proxy: Even without blood pressure readings or BMI, the models achieved an Area Under the Curve (AUC) of 0.74. This is comparable to models that do use clinical data, proving that social determinants and history of service use are powerful predictors of health outcomes.
- The "Fairness" Win-Win: One of the most significant findings was that there was no trade-off between model performance and equity. Improving the model for the whole population generally improved it for sub-groups, and vice versa.
- Bigger is Better for Training: Interestingly, Scenario C (training on specific ethnic groups) performed worse than using the whole population. This suggests that the "shared" patterns of disease progression across the entire population provide more signal than the specific "unique" patterns within a smaller ethnic subset.
- Non-Clinical Variables Matter: "Geographical area" and "Income level" appeared in the top 10 most important variables for the Random Forest model, yet these are almost never included in traditional clinical risk calculators.
Key Results
- Top Performer: Xgboost was the clear winner, outperforming Random Forest by about 6.4% for the total population and up to 10% for the Pasifika population.
- Predictive Power: The Xgboost model achieved an average AUC of 0.74 for predicting CVD events three years into the future.
- Variable Importance: The top predictors were Age, Geographical area, Hospitalization history, and Years living with diabetes.
- Stability: The models remained stable when tested on the validation dataset (2016–2018), showing that the patterns the ML learned weren't just "noise" from a specific year.
Limitations and Future Work
The authors acknowledge a few hurdles:
- Missing Clinical Data: While the model performed well without it, they expect that adding clinical features (like HbA1c levels) would further boost accuracy.
- Computational Barriers: Running these models on massive, government-held datasets requires significant computing infrastructure that isn't always accessible for routine health planning.
- Ethnicity Misclassification: There is a known risk of Māori being misclassified as non-Māori in administrative data, which could skew equity results.
Future Work: The researchers suggest expanding the feature set to investigate "unknown" CVD risk factors and applying this same methodology to other chronic conditions like cancer or kidney disease. They also emphasize that the goal isn't just better math, but better resource allocation to the groups the model identifies as being at the highest risk.
Summary updated