The maximum entropy principle for compositional data

Corey Weistuch, Jiening Zhu, Joseph O. Deasy, Allen R. TannenbaumBMC Bioinformatics 2022 · DOI

TL;DR

This paper introduces Compositional Maximum Entropy (CME), a novel data-driven framework for understanding complex systems where components' proportions are fixed (compositional data). CME infers underlying relationships by maximizing entropy, offering a more general and interpretable approach than existing methods, as demonstrated in microbiome and cancer gene expression analyses.

What Problem Does This Paper Address?

Many real-world systems, from cellular protein abundances to microbial communities and chemical reactions, are "compositional." This means the proportions of their parts always sum to a fixed total (e.g., 100%). Analyzing these systems is challenging for two main reasons:

  1. Complexity and Stochasticity: The behavior of the whole system emerges from the interactions of many individual components, often in a probabilistic manner.
  2. Simplicial Geometry: The constraint that proportions must sum to one means the data lie on a mathematical structure called a simplex. This geometric constraint can lead to spurious correlations when traditional statistical methods are applied, and existing methods often struggle with zero values or require transformations that introduce biases.

The authors aim to develop a general and principled approach to infer the underlying multivariate relationships within these compositional systems directly from data, overcoming the limitations of current methods.

Methods

The core of the paper is the Compositional Maximum Entropy (CME) framework. This approach leverages the principle of maximum entropy, which states that the most unbiased model is the one that maximizes entropy subject to known constraints.

Here's how it works:

  • Input: Observed relative abundances of components (e.g., species in a microbiome, genes in a cell).
  • Constraints: The model is constrained by the estimated first moments (means) and second moments (covariances) of the data, along with the inherent compositional constraint (sum to 1).
  • Maximizing Entropy: CME finds a probability distribution over the possible states of the system that maximizes entropy while satisfying these constraints. This results in a model that is as "uninformative" as possible beyond what the data explicitly tell us.
  • Interpretable Parameters: The resulting CME model yields two key interpretable parameters:
  • h_i (Influence weights): These represent the influence of each individual component on the system. Higher h_i values suggest a component tends to have higher abundances.
  • K_ij (Interaction strengths): These quantify the pairwise interactions between components. Positive K_ij suggests coexistence, while negative K_ij indicates mutual exclusion.
  • Computational Approach: Fitting these models is computationally challenging due to the intractable integral over the simplex. The authors adapt a pseudolikelihood approximation and use a scalable L-BFGS algorithm for convex optimization, enabling efficient fitting even for larger networks.

Datasets

  • Simulated Lotka-Volterra (cLV) model data: Used to demonstrate CME's ability to recover known inter-species interactions under different ecological regimes (mutualism, neutralism, competition).
  • METABRIC dataset: Microarray gene expression data for triple-negative breast tumors (1904 samples, 24368 genes). This dataset was used to analyze gene-gene interactions in cancer.
  • Human Protein Reference Database (HPRD): Used to obtain a curated interaction network of human proteins to identify relevant genes for the cancer analysis.

Key Insights and Ideas

  • Data-Driven Inference: CME learns the model structure directly from the data, making minimal assumptions beyond what is supported by observations.
  • Interpretability: The h_i and K_ij parameters provide intuitive insights into component influences and pairwise interactions, which is often lacking in other compositional data analysis methods.
  • Generality: CME is a flexible framework that can incorporate various constraints beyond simple means and covariances, making it applicable to a wide range of compositional systems.
  • Handling Compositionality: CME inherently accounts for the simplex geometry and the fixed-sum constraint, avoiding spurious correlations and biases introduced by transformations.
  • Biological Relevance: The method successfully identified key drivers and interactions in both ecological (microbiome) and biomedical (cancer) contexts, aligning with known biological principles.

Key Results

  • Microbiome Analysis: CME accurately inferred inter-species interactions from simulated Lotka-Volterra data, correctly distinguishing between mutualism, neutralism, and competition regimes, even with limited data. The K_ij parameter directly mapped to the interaction coefficients in the cLV model.
  • Cancer Gene Expression Analysis:
    • CME identified a tightly connected network of 17 key genes in triple-negative breast cancer.
  • It revealed unequal influences (h_i) among these genes, highlighting known cancer drivers like SRC and TP53.
  • The inferred interactions (K_ij) provided an intuitive explanation for gene clustering and critical balances (e.g., growth vs. damage repair between SRC and TP53).
  • CME outperformed a common alternative (logit-normal distribution) in terms of interpretability and capturing key biological relationships, particularly the negative covariance between SRC and TP53.

Limitations and Future Work

  • Network Size: While the pseudolikelihood approach is scalable, the paper primarily demonstrates CME on relatively small networks. Extension to very large networks (thousands of nodes) might require further optimization or regularization strategies.
  • Higher-Order Moments: The current implementation focuses on first and second moments. Future work could explore incorporating higher-order moments or other domain-specific constraints to capture more complex relationships.
  • Comparison to Pseudolikelihood: The paper mentions that score matching approaches can be viewed as approximations of pseudolikelihood, suggesting a potential avenue for future research to explore the relationship between CME and these methods.
  • Generalization: The authors posit that CME can be easily generalized to incorporate new types of constraints and data, opening doors for applications in diverse fields.

Summary updated

BibTeX
@article{weistuch2022maximum,
  author = {Corey Weistuch and Jiening Zhu and Joseph O. Deasy and Allen R. Tannenbaum},
  title = {{The maximum entropy principle for compositional data}},
  journal = {BMC Bioinformatics},
  year = {2022},
  doi = {10.1186/s12859-022-05007-z},
}
Figure