Geometry Preserving Analysis of Sparse Compositional Data

40 concepts28 papers1 proposal

The Geometry of Proportions and the Zero Problem

Compositional data represents parts of a whole, such as the relative abundance of microbes in a gut sample, the proportions of immune cells in a tumor, or the distribution of visual words in an image histogram. Because these components are non-negative and sum to a fixed constant (like 1 or 100%), they do not live in standard Euclidean space. Instead, they are constrained to a bounded geometric space known as the simplex.

To apply standard statistical tools like linear regression, classification, or principal component analysis (PCA), researchers traditionally map the simplex into Euclidean space using log-ratio transformations (such as the isometric log-ratio or centered log-ratio). The intuition is simple: taking ratios removes the constant-sum constraint, and taking logarithms maps the bounded proportions to the entire real line.

However, this elegant mathematical bridge collapses when encountering a common feature of real-world data: zeros. Whether a zero represents a true absence (an "essential zero") or a value below the detection limit, the logarithm of zero is mathematically undefined. As a result, standard log-ratio transformations cannot natively process sparse datasets.

Why Pseudocounts Distort Simplex Geometry

The traditional workaround for the zero problem is the "pseudocount heuristic"—adding a small positive constant ($\epsilon$) to all zero entries or performing a multiplicative substitution. While computationally convenient, this approach introduces severe geometric and statistical distortions.

To understand why, we must look at the geometry of the log-ratio transformation. As a proportion approaches zero, its log-ratio approaches negative infinity. This means that log-ratio methods effectively treat the boundary of the simplex as being at an infinite distance from the center.

When a researcher replaces a zero with a tiny pseudocount, they are placing that data point extremely far out on the edge of the Euclidean space. As highlighted by Junyoung Park et al. in Kernel Methods for Radial Transformed Compositional Data with Many Zeros, this geometric stretching amplifies minor fluctuations near the boundary. A tiny change in the chosen value of $\epsilon$ can unpredictably warp the structural relationships between data points, leading to erratic and unstable analytical outcomes.

Furthermore, as noted by Marta Avalos et al. in Representation Learning of Compositional Data and George Armstrong et al. in Applications and Comparison of Dimensionality Reduction Methods for Microbiome Data, these ad-hoc adjustments lack a principled statistical foundation and frequently introduce downstream mathematical bias, skewing biological or physical interpretations.

Bypassing Log-Ratios via Kernel Spaces

To resolve this zero-value sensitivity without distorting the data, recent research focuses on transformation-free or geometry-preserving frameworks. Instead of forcing the data into Euclidean space via logarithms, these methods operate directly on the native geometry of the data or map it to alternative spaces where zeros are well-behaved.

One prominent strategy is to use kernel methods. By mapping compositional data into a reproducing kernel Hilbert space (RKHS), models can capture complex, non-linear relationships without ever computing a log-ratio. For instance, Junyoung Park et al. in Kernel Sufficient Dimension Reduction and Variable Selection for Compositional Data via Amalgamation demonstrate that operating directly in kernel space bypasses the mathematical difficulties of zero values entirely. They introduce an "amalgamation" technique that groups unselected sparse variables into a single category, maintaining statistical consistency without artificial data imputation.

Similarly, Junyoung Park et al. in Kernel Methods for Radial Transformed Compositional Data with Many Zeros propose a radial transformation that maps compositional data onto a hypersphere rather than an infinite Euclidean space. Because the boundary of a hypersphere remains at a finite distance from its center, this approach preserves the true geometry of sparse datasets and eliminates the need for arbitrary "fudge factors."

Redefining Probability Distributions on the Simplex

Another frontier in zero-value handling involves developing new probabilistic models that are natively defined on the boundaries of the simplex. Traditional distributions used for compositional data, such as the Dirichlet distribution, suffer from singularities or are undefined when components equal zero.

To address this, Elliott Gordon-Rodríguez et al. in The continuous categorical: a novel simplex-valued exponential family introduced the Continuous Categorical (CC) distribution. Unlike the Dirichlet distribution, the CC distribution handles zero values gracefully and remains well-defined at the boundaries of the simplex. This allows probabilistic and deep learning models to learn from sparse compositional vectors natively, producing unbiased estimates without requiring artificial noise or offsets.

Nonparametric and Rank-Based Alternatives

In fields like metagenomics and immunology, where sparsity is exceptionally high, researchers are also turning to rank-based and nonparametric methods that avoid log-ratios entirely.

For example, Bo Yuan in RSim: A reference-based normalization method via rank similarity introduces a normalization method that leverages rank similarity to identify stable, non-differential features. By using these stable features as an internal mathematical reference, the method corrects for technical biases and handles zero counts without needing pseudocounts.

In network inference, Grace Yoon et al. developed SPRING (Microbial Networks in SPRING - Semi-parametric Rank-Based Correlation and Partial Correlation Estimation for Quantitative Microbiome Data), which uses a semi-parametric rank-based correlation estimator within a truncated Gaussian copula. This framework naturally accommodates the excess zeros typical of microbial counts, avoiding the significant biases that pseudocounts introduce into network structures.

By shifting away from log-ratios and toward geometry-preserving transformations, kernel spaces, and boundary-aware distributions, modern compositional data analysis treats zeros not as mathematical errors to be patched, but as meaningful geometric features of the data.

Key Foundational Work

Causal discovery with scale-mixture model for spatiotemporal variance dependencies (2012) — contributed 1 concept elements to this direction.
Optimization for Hue Constant RGB Sensors (2002) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 29 papers
  2. Version 1 · 2026-10-04 · 41 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_zero_value_handling_in_compositional_dat_3a18cc71,
  title = {zero-value handling in compositional data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-value-handling-in-compositional-data/rdr_3a18cc7192f54002a131293d38ea5f38},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_zero_value_handling_in_compositional_dat_3a18cc71_v2,
  title = {zero-value handling in compositional data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/zero-value-handling-in-compositional-data/rdr_3a18cc7192f54002a131293d38ea5f38/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake