Studies linking environmental exposures to metabolic disease use exposures that are mostly geographic. Outdoor light at night, measured from satellites, and fine particulate matter are both highest in cities. Cities are also where people live close to clinics and are more likely to have had their diabetes or hypertension diagnosed. When a risk model is trained on "has a doctor ever told you that you have diabetes," a steeper disease curve at high light exposure could mean the light harms metabolism. It could equally mean that brightly lit places diagnose more of their existing disease. A model fitted to the whole population blends these two stories into one curve. It will then systematically under-predict true risk in poorly served rural areas, which are exactly the places where a screening model is most needed.
This note proposes to pull the two stories apart. The method has two parts. The first compares outcomes defined by self-reported diagnosis against outcomes defined by measured biomarkers (fasting glucose, HbA1c, measured blood pressure) in the same people. The gap between the two is undiagnosed disease, and its size tells us how much of an exposure's apparent effect is really an access effect. The second part fits exposure–response surfaces separately by healthcare-access regime, while borrowing strength across regimes. A held-out "no-harm" test then checks whether giving each regime its own model actually helps that regime compared with a single shared model. The claim is empirical and falsifiable. We predict that the association between light at night and diagnosed diabetes is measurably stronger than its association with biomarker-defined diabetes, and that this gap shrinks once access to care is modeled. If the gap is absent, the concern is laid to rest, and the published exposure curves are more trustworthy than we suspect.
Why this gap exists in the current literature
The motivating evidence comes from the China Health and Retirement Longitudinal Study (CHARLS), a national survey of adults aged 45 and over. The CHARLS light-at-night study matched satellite light intensity to 11,729 participants. It fit smooth dose–response curves (restricted cubic splines) within a single pooled logistic regression. It found higher diabetes, metabolic syndrome, overweight and dyslipidemia at higher light. It also found a dip in obesity risk at the brightest quartile, which the authors attributed to better urban healthcare and infrastructure. That explanation is a hypothesis about access, and the study never measured access. This proposal is built to test it.
The CHARLS PM2.5 study went further toward causality. It used weather variables such as wind, rain and thermal inversions as instruments, meaning sources of pollution variation unrelated to individual behavior. It reported unexpected directional results, including triglycerides falling as PM2.5 rose, and an interaction in which high-intensity exercise worsened the waist-circumference effect of pollution. Its models are linear with hand-specified product terms, and it does not measure healthcare access.
On the US side, the NHANES environmental-chemicals study showed that non-linear classifiers outperform linear models on a mixture of 23 blood and urine toxicants. Its support vector machine reached AUC 0.822 for hypertension. It trained one pooled model, and it imputed missing values before splitting train from test, a leak the paper itself flags.
These three papers motivate the proposal: each finds an exposure signal, and none can say whether that signal passes through access to diagnosis.
Two other lines of work constrain the design. The Boston Medical Center hypertension study is the closest existing work. It trained race-specific gradient-boosted models on social-needs and census features, and found that transport and food insecurity dominate for Black patients. It then equalized false-negative rates by shifting decision thresholds after training. It has no physical exposures, and it covers a single urban center. A scoping review of population-level NCD models found that three quarters of such models ignore bias entirely, and that mitigation, where it appears, is almost always a sex adjustment rather than attention to socioeconomic or geographic structure.
Finally, two papers supply methods. Fairness without Harm formalizes when separate per-group models are justified. "Rationality" requires that every group does at least as well under its own model as under the pooled one. "Envy-freeness" requires that no group would do better using another group's model. The paper checks these with held-out McNemar tests, but its exact mixed-integer solver only handles linear models on small data. Demographic-Agnostic Fairness without Harm learns latent groups when group labels are unavailable. We reuse the held-out harm test from the first and the latent-group learner as a competitor. We do not reuse the mixed-integer solver.
Core Claim
Let $x$ be a person's exposures and covariates, $B$ the biomarker-defined disease indicator, and $D$ the self-reported diagnosis. Diagnosis requires both having the disease and having it detected, so
$$P(D=1 \mid x) = P(B=1 \mid x)\, P(D=1 \mid B=1, x).$$
The first factor is the biology the exposome literature wants to estimate. The second factor, the detection rate, is set largely by access to care. Any exposure that tracks urbanicity, such as light at night, PM2.5 or traffic-related hydrocarbons, will look more harmful on the $D$ scale than on the $B$ scale wherever detection rises with urbanicity.
The claim has three parts. First, this distortion is large enough to change published exposure–response shapes. Second, it is concentrated where access is poor. Third, models trained on $D$ are miscalibrated against true risk in low-access strata in a way that pooled, access-blind evaluation hides. The contribution is not a new learner. It is a measurement design that makes the detection factor visible, together with an honest test of whether access-specific modeling adds anything beyond a well-tuned pooled model.
Research Program
The primary cohort is CHARLS. It records both self-reported physician diagnoses and measured blood biomarkers and blood pressure in the same people, across the 2011 and 2015 waves. That makes it possible to compute $B$, $D$, and the undiagnosed fraction for diabetes, hypertension and dyslipidemia. Exposures are satellite light at night, following the CHARLS light study's processing of DMSP-OLS and SNPP-VIIRS data, and PM2.5 with its meteorological instruments, following the PM2.5 study.
Access is the weakest data link, and we state that plainly. The research file does not document facility-distance variables in CHARLS. Access will therefore be built at the finest geography the restricted CHARLS release permits. The layers are urban versus rural residence; road-network driving distance from the community to the nearest county hospital, computed as in the US driving-distance study, which also reminds us that distance does not guarantee quality of care; and individual insurance and utilization. If location resolution is too coarse for a network-distance layer, the study falls back to rural/urban residence and utilization alone, and says so.
The US replication uses NHANES, with measured versus self-reported hypertension and diabetes and the chemical panels from the NHANES studies. NHANES public files lack fine geography, so access there is individual-level: usual source of care and insurance status. This tests whether the decomposition holds when "access" is personal rather than spatial.
The analysis has three stages. Stage one estimates exposure–response curves on the $B$ and $D$ scales, overall and within access strata. It uses both pooled gradient-boosted trees, summarized with accumulated-local-effect curves, and spline logistic regression, so that the conclusion does not depend on the model class. Stage two fits the detection model $P(D=1 \mid B=1, x)$ directly among biomarker-positive people, and asks how much of the exposure effect it absorbs. Stage three builds prediction models for $B$ and compares pooled, fully separate, and partially pooled access-specific models.
In the partially pooled version, a shared boosted model is trained first. Each access stratum then fits a small, shrunken correction on the shared model's residuals. This is our scalable substitute for the mixed-integer decoupling in Fairness without Harm, because tree ensembles are exactly where that method cannot go.
Throughout, imputation happens inside training folds. Cross-validation holds out whole communities, so that neighbors do not leak spatial information between train and test. Survey weights are applied in estimation of curves and prevalences.
Baselines
The naive reference point is a pooled gradient-boosted model trained on self-reported diagnosis with no access features. This is how an off-the-shelf exposome screening model would be built. We report its calibration against biomarker-defined disease in each access stratum. If it is already well calibrated in low-access strata, the core concern fails.
The strongest existing predictive competitor is a pooled gradient-boosted model trained on biomarker labels, with access variables and exposure-by-access interactions supplied as features. We select those interactions using the split-gain-plus-SHAP pruning of Interaction-Enhanced and Explainable Machine Learning for Diabetes Risk Prediction, which found that adding every pairwise interaction hurt generalization. Trees already model interactions, so this is the model most likely to make stratum-specific models unnecessary.
For a published number, the closest comparable is the NHANES chemicals study's SVM: AUC 0.822, accuracy 0.751, recall 0.717 for hypertension. That figure was obtained with imputation before the split, so we re-run the same feature set with in-fold imputation and report both. For context on social-feature-only models, the Boston study reports an XGBoost AUC of 74.49% for uncontrolled hypertension. We found no reported result on Lacuna for predictive AUC of metabolic outcomes in CHARLS, so the CHARLS baselines are established by this study.
The remaining comparisons are these:
- a threshold-shifting baseline, following the Boston study, that tunes a separate decision cutoff per access stratum;
- fully separate per-stratum models;
- the latent-group learner from Demographic-Agnostic Fairness without Harm, with the number of groups matched to the number of access strata;
- stratified spline logistic regression, the classical epidemiology answer.
The ablation that isolates the key ingredient is the same partially pooled model trained on $D$ instead of $B$. It shows how much of any gain comes from the label rather than from the stratification.
Every model gets the same features, the same community-held-out folds, and the same hyperparameter search budget, measured in number of trials. Beating the strong baseline means the following. The partially pooled model passes the held-out rationality test in every access stratum. It improves the log-loss or calibration of the lowest-access stratum by a margin whose paired bootstrap interval excludes zero. It does this without losing AUC in any other stratum.
Why not just…?
Why not just give one gradient-boosted model the access variables and let the trees find the interactions? This is the real threat, and it may win on prediction. Boosted trees split on rurality and then on exposure without being told to. Stratum-specific corrections only help when a stratum is both distinct and small enough that shared splits underfit it. Stage three settles the question directly. If the pooled model with access features matches the partially pooled one in every stratum, we report that the stratification is unnecessary. The contribution then rests on the label decomposition, which no amount of feature engineering on $D$-labels can reproduce, because the error sits in the label, not the features.
Why not just use classical stratified spline regression, as epidemiologists already do? For two exposures in CHARLS this may be enough, and we include it as a baseline. It becomes fragile with the NHANES chemical mixtures, where the NHANES chemicals study found linear models explained as little as 5% of diastolic variance. It also gives no out-of-sample harm test. If splines and boosted trees give the same $B$-versus-$D$ curve gap, that agreement strengthens the scientific finding rather than weakening it.
Why not just train on biomarker labels and forget about access? Switching to $B$ is the fix we expect to recommend, but by itself it leaves two questions unanswered. It does not show how much of the published literature, built on diagnosis-based outcomes, is distorted. It also does not show whether the exposure curves still differ by access regime once labels are clean. They might, through treatment effects, which is what the CHARLS light study's obesity dip suggests. The decomposition answers both questions, and the $D$-trained ablation measures the cost of not switching.
Why not just shift decision thresholds per group, as the Boston study did? Threshold shifting equalizes one error rate at one operating point. It cannot repair a curve whose shape is wrong, or probabilities that are miscalibrated across the exposure range. Those are exactly the failures predicted here. The comparison will show this through calibration curves stratified by exposure quintile within each access stratum.
Decisive Checks
Three results decide the proposal.
The first test is scientific. In CHARLS, the fitted diagnosed-diabetes-versus-light curve should be steeper than the biomarker-defined curve, with a bootstrap interval on the difference in slope between the top and bottom light quartiles that excludes zero. That difference should shrink by at least half once the detection model conditions on access. If the $B$ and $D$ curves coincide, the access-confounding concern is refuted for light at night, and we would report that as a useful negative result. The same test is then repeated for PM2.5, for hypertension, and in NHANES, using personal access in place of geographic access.
The second test concerns harm to underserved groups. The naive pooled $D$-model should show a calibration-in-the-large deficit against $B$ in the lowest-access stratum, meaning its average predicted risk falls below the observed rate. That deficit should be larger than in the highest-access stratum.
The third test is the modeling question: whether any access-specific model passes the rationality test and beats the pooled model with access features in the lowest-access stratum. We commit in advance to reporting whichever way this third check falls. The first two checks carry the proposal on their own.
References
- Association between outdoor artificial light at night and metabolic diseases in middle-aged to older adults—the CHARLS survey
- Causal association between PM2.5 and metabolic syndrome in the Chinese elderly population-insights from a cohort study of CHARLS
- Environmental chemical exposures and a machine learning-based model for predicting hypertension in NHANES 2003–2016
- Accounting for racial bias and social determinants of health in a model of hypertension control
- Bias in machine learning applications to address non-communicable diseases at a population-level: a scoping review
- Fairness without Harm: Decoupled Classifiers with Preference Guarantees
- Demographic-Agnostic Fairness without Harm
- Income disparities in driving distance to health care infrastructure in the United States: a geographic information systems analysis
- Interaction-Enhanced and Explainable Machine Learning for Diabetes Risk Prediction