Supervised learning and model analysis with compositional data

Shimeng Huang, Elisabeth Ailer, Niki Kilbertus, Niklas Pfister

TL;DR

KernelBiome is a kernel-based nonparametric machine learning framework tailored to sparse, compositional data such as microbiome sequencing counts. It unifies non-linear prediction, data-adaptive kernel selection, and prior biological knowledge (like phylogenetic trees) with two novel, theoretically consistent interpretability metrics—Compositional Feature Influence (CFI) and Compositional Feature Dependence (CPD)—that strictly respect simplex geometry.

What Problem Does This Paper Address?

High-throughput sequencing data (e.g., marker-gene or metagenomic profiles) are inherently compositional: they measure relative abundances constrained to sum to one (lying on the standard simplex $\mathcal{S}^{p-1}$). This introduces spurious negative correlations and complex inter-dependencies. Furthermore, sequencing data are severely zero-inflated.

Traditional compositional techniques rely on linear log-contrast models (using log-ratio transforms). These fail in practice because:

  1. Logarithms of zero are undefined, forcing practitioners into ad-hoc pseudo-count imputation.
  2. Linear models cannot capture complex biological interactions or presence/absence zero-patterns.

Conversely, while black-box machine learning methods (such as Random Forests or standard SVMs) can learn non-linear patterns, standard explainability tools (like Friedman's partial dependence plots or partial derivatives) perturb single features in isolation, inadvertently pushing points off the simplex and invalidating their biological interpretations.

Methods

KernelBiome casts supervised regression and classification onto the simplex using reproducing kernel Hilbert spaces (RKHS) and kernel ridge regression / support vector machines:

  • Simplex-Targeted Kernels: Explores four kernel families adapted to compositional constraints:
  • Euclidean: Linear and universal RBF kernels restricted to the simplex.
  • Aitchison Geometry: Reproduces log-contrast models while turning zero-replacement pseudo-counts into a cross-validated hyperparameter.
  • Probability Distributions: Generalized Jensen-Shannon and Hilbertian metrics that naturally tolerate exact zeros.
  • Riemannian Manifolds: Heat-diffusion kernels on multinomial statistical manifolds.
  • Incorporating Prior Biological Knowledge: A formulation for positive-semidefinite weight matrices $W$ (e.g., phylogenetic UniFrac distances) to construct weighted kernels: $$k_W(x, y) = \sum_{i,j=1}^p W_{ij} \, k_0(x_i, y_j)$$
  • Compositionally Valid Interpretability Metrics:
  • CFI (Compositional Feature Influence): Evaluates expected derivatives under multiplicative coordinate rescaling $\psi_j(x, c)$ while normalizing the remainder back to the simplex. For linear log-contrast models, CFI provably reduces to the model's $\beta$-coefficients.
  • CPD (Compositional Feature Dependence): Evaluates predictions when a coordinate is fixed to $z$ and others are renormalized.
  • Consistency Guarantees: The authors prove that empirical Monte Carlo estimates of both CFI and CPD converge in probability to their true population quantities.
  • Downstream Distance-Based Analyses: Uses the kernel-induced semi-metric to conduct kernel PCA and formulate scalar summary scores (e.g., data-adaptive $\alpha$-diversity or "health distance" to a healthy centroid).
  • Implementation: Built in Python using JAX (with JIT compilation for GPU/TPU acceleration) and scikit-learn.

Datasets

  • 33 Public Microbiome Datasets: Evaluated across 9 regression and 24 classification tasks spanning human gut metagenomics, environmental soil, and ocean surveys (including well-known benchmarks like cirrhosis, centralparksoil, uk, hiv, and tara).
  • Semi-Synthetic Benchmark: Derived from the uk dataset (327 species, $n = 882$) to benchmark weighted kernels against known ground-truth tree groupings under informative vs. adversarial configurations.

Key Insights and Ideas

  • Pseudo-counts as Learnable Hyperparameters: Instead of fixing an arbitrary pseudo-count before training, tuning the shift parameter inside the Aitchison kernel significantly improves out-of-sample prediction.
  • Strict Simplex Perturbations for Explainability: Evaluating model gradients or partial dependencies on compositions requires simultaneously adjusting all remaining coordinates; CFI and CPD resolve the mathematical invalidity of standard post-hoc explanation techniques.
  • Prior Knowledge is a Double-Edged Sword: Weighting kernels by phylogenetic proximity boosts statistical power when true associations align with phylogenies, but actively hurts predictive performance when signals cross-cut taxonomy.
  • Unified Bridge to Classical Statistics: Under the unweighted or block-weighted Aitchison kernel, KernelBiome mathematically reduces to standard or tree-aggregated log-contrast models, ensuring zero performance penalty when the true underlying mechanism is simple and linear.

Key Results

  • Top-Tier Predictive Performance: Across 33 benchmark tasks, KernelBiome consistently matched or outperformed baseline competitors. Based on Wilcoxon signed-rank tests, KernelBiome was significantly outperformed in only 51.5% of pairwise comparisons, compared to 66.7% for Random Forests, 78.8% for $\ell_1$-penalized log-contrast models (LogCont-L1), and 84.9% for standard SVM-RBF.
  • Value of Prior Structure: In semi-synthetic experiments with informative phylogenetic weights, weighted KernelBiome outperformed unweighted models ($p = 2.34 \times 10^{-18}$); mismatched adversarial weights significantly deteriorated performance ($p = 1.25 \times 10^{-9}$).
  • Case Study Utility: On the liver cirrhosis dataset ($n = 130$), KernelBiome correctly clustered taxa by phylum in CFI space and derived a kernel-distance-to-healthy score that separated healthy from cirrhotic cohorts more cleanly than classical Simpson diversity.

Limitations and Future Work

  • Computational Cost: Evaluating nested cross-validation across 55 kernel configurations and hyperparameter grids requires $O(KH N_{\text{in}} N_{\text{out}} n^3)$ operations (over 27,000 model fits in the default grid), making it computationally heavy on large sample sizes without pre-filtering kernels.
  • Off-Support Extrapolation for CPD: Compositional feature dependence assumes the perturbation stays within the data support; if combinations are unobserved in nature, interpreting CPD requires caution.
  • Feature Screening Overhead: High-dimensional metagenomic analyses (thousands of OTUs/species) require initial screening (e.g., top 50 taxa via CFI) to manage visualization and runtime, which can risk overfitting if not validated via nested subsampling.

Summary updated

Abstract

Author summary In recent years, advances in gene sequencing technology have allowed scientists to examine entire microbial communities within genetic samples. These communities interact with their surroundings in complex ways, potentially benefiting or harming the host they inhabit. However, analyzing the microbiome—the measured microbial community—is challenging due to the compositionality and sparsity of the data. In this study, we developed a statistical framework called KernelBiome to model the relationship between the microbiome and a target of interest, such as the host’s disease status. We utilized a type of machine learning model called kernel methods and adapted them to handle the compositional and sparse nature of the data, while also incorporating prior expert knowledge. Additionally, we introduced two new measures to help interpret the contributions of individual compositional components. Our approach also demonstrated that kernel methods increase interpretability in analyzing microbiome data. To make KernelBiome as accessible as possible, we have created an easy-to-use software package for researchers and practitioners to apply in their work.
BibTeX
@article{huang2023supervised,
  author = {Shimeng Huang and Elisabeth Ailer and Niki Kilbertus and Niklas Pfister},
  title = {{Supervised learning and model analysis with compositional data}},
  journal = {PLOS Computational Biology},
  year = {2023},
  doi = {10.1371/journal.pcbi.1011240},
}

Loading related content...

Figure