Using permutations to detect, quantify and correct for confounding in machine learning predictions

Elias Chaibub NetoarXiv 2018 · DOI · arXiv

Abstract

Clinical machine learning applications are often plagued with confounders that are clinically irrelevant, but can still artificially boost the predictive performance of the algorithms. Confounding is especially problematic in mobile health studies run "in the wild", where it is challenging to balance the demographic characteristics of participants that self select to enter the study. An effective approach to remove the influence of confounders is to match samples in order to improve the balance in the data. The caveat is that we end-up with a smaller number of participants to train and evaluate the machine learning algorithm. Alternative confounding adjustment methods that make more efficient use of the data (e.g., inverse probability weighting) usually rely on modeling assumptions, and it is unclear how robust these methods are to violations of these assumptions. Here, rather than proposing a new approach to prevent/reduce the learning of confounding signals by a machine learning algorithm, we develop novel statistical tools to detect, quantify and correct for the influence of observed confounders. Our tools are based on restricted and standard permutation approaches and can be used to evaluate how well a confounding adjustment method is actually working. We use restricted permutations to test if an algorithm has learned disease signal in the presence of confounding signal, and to develop a novel statistical test to detect confounding learning per se. Furthermore, we prove that restricted permutations provide an alternative method to compute partial correlations, and use this result as a motivation to develop a novel approach to estimate the corrected predictive performance of a learner. We evaluate the statistical properties of our methods in simulation studies.

Introduction

Machine learning models are inherently "lazy"—they will find the easiest path to a correct prediction. In clinical studies, especially those conducted "in the wild" via smartphones, participant groups are often unbalanced. For example, if a Parkinson’s disease study has older patients and younger healthy controls, a model might achieve high accuracy simply by becoming an "age detector" rather than a "Parkinson’s detector." This phenomenon, known as confounding, severely limits the generalizability of clinical ML models.

Causal structures representing confounding (a), selection bias (b), and undirected associations (c).

Standard mitigation strategies like matching (discarding data to balance groups) or weighting (relying on modeling assumptions) have significant drawbacks. There is a critical need for diagnostic tools that can detect and quantify confounding without requiring data removal or rigid distributional assumptions.

Methodology: Restricted vs. Standard Permutations

The core innovation of this work is a statistical framework based on restricted permutations. Unlike standard permutations, which shuffle labels across the entire dataset to create a baseline for random guessing, restricted permutations shuffle labels only within specific levels of a confounder (e.g., within gender or age groups).

Visualization of the permutation methodology. (a-c) Causal graphs for original data, restricted permutations, and standard permutations. (d-f) Feature matrix X, confounder C, and response Y. (g-h) Comparison of restricted permutations (Y*) vs standard permutations (Y**).
  1. Standard Permutations ($Y^{**}$): Shuffling labels across the entire dataset destroys all associations between features ($X$) and labels ($Y$).
  2. Restricted Permutations ($Y^*$): Shuffling labels only within levels of a confounder ($C$) destroys the direct link between features and disease but preserves the association between the confounder and the disease.
  3. The Correction Formula: The paper proves that the difference between these two permutation distributions can be used to calculate partial correlations. A Gaussian mapping technique is then used to "shift" the observed performance metric (e.g., AUC) back to a "corrected" value ($m_c$), effectively removing the confounding bias.
  4. Confounding Test: A new hypothesis test is proposed to formally determine if an algorithm has learned the confounding signal.

Experimental Evaluation

The methods were evaluated using synthetic datasets designed to mimic a Parkinson’s disease diagnostic task. Data was generated using causal diagrams where a binary confounder ($C$) influences both features ($X$) and disease status ($Y$). A Latin Hypercube design was used to sweep through various strengths of confounding and disease signals across 1,000 simulated scenarios.

Key Results and Insights

  • Detection Power: In simulations, the proposed confounding test showed high power to detect when a model was learning from confounders, even with subtle effects.
  • Accurate Correction: The corrected AUC ($m_c$) successfully recovered the true disease signal. In cases where the model only learned the confounder, the corrected AUC dropped to 0.5 (random guess), even if the raw observed AUC was high (e.g., 0.75).
  • Theoretical Bridge: The work bridges permutation-based ML evaluation and classical statistics by proving that restricted permutations are an alternative way to compute partial correlations.
  • Asymptotic Normality: For test sets of reasonable size ($n \ge 30$), the permutation distributions follow a normal distribution, allowing for fast analytical approximations instead of computationally expensive shuffling.

Conclusion

This framework provides a powerful post-hoc diagnostic tool for clinical machine learning. While it does not "un-teach" biased signals during training, it allows for honest reporting by quantifying exactly how much a model's performance is driven by confounding factors. Future work will explore using these tools to evaluate the effectiveness of other adjustment methods like inverse probability weighting.

Summary updated

Abstract

Clinical machine learning applications are often plagued with confounders that are clinically irrelevant, but can still artificially boost the predictive performance of the algorithms. Confounding is especially problematic in mobile health studies run "in the wild", where it is challenging to balance the demographic characteristics of participants that self select to enter the study. An effective approach to remove the influence of confounders is to match samples in order to improve the balance in the data. The caveat is that we end-up with a smaller number of participants to train and evaluate the machine learning algorithm. Alternative confounding adjustment methods that make more efficient use of the data (e.g., inverse probability weighting) usually rely on modeling assumptions, and it is unclear how robust these methods are to violations of these assumptions. Here, rather than proposing a new approach to prevent/reduce the learning of confounding signals by a machine learning algorithm, we develop novel statistical tools to detect, quantify and correct for the influence of observed confounders. Our tools are based on restricted and standard permutation approaches and can be used to evaluate how well a confounding adjustment method is actually working. We use restricted permutations to test if an algorithm has learned disease signal in the presence of confounding signal, and to develop a novel statistical test to detect confounding learning per se. Furthermore, we prove that restricted permutations provide an alternative method to compute partial correlations, and use this result as a motivation to develop a novel approach to estimate the corrected predictive performance of a learner. We evaluate the statistical properties of our methods in simulation studies.
BibTeX
@misc{neto2018using,
  author = {Elias Chaibub Neto},
  title = {{Using permutations to detect, quantify and correct for confounding in machine learning predictions}},
  year = {2018},
  eprint = {1805.07465},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1805.07465},
}
Figure