Vector Generalizations of Entropy Power Inequalities

16 concepts14 papers2 proposals

In classical information theory, the Gaussian distribution is the "gold standard" for extremal behavior. It maximizes entropy for a given variance and minimizes Fisher information for a given covariance. These properties are encapsulated in the Entropy Power Inequality (EPI), which provides a fundamental lower bound on the entropy of the sum of independent random variables.

However, as communication systems moved from single-antenna (scalar) to multi-antenna (MIMO) and multi-sensor (vector) configurations, the scalar EPI became insufficient. The transition from scalar variances to the complex geometry of positive semidefinite covariance matrices introduces significant analytical hurdles. A growing body of research focuses on generalizing these extremal inequalities to multivariate settings to characterize the fundamental limits of vector-valued communication.

The Geometry of Vector Uncertainty

The core difficulty in moving from scalar to vector sources is that uncertainty is no longer a single number; it is a shape. In one dimension, the "size" of noise is its variance. In multiple dimensions, noise is characterized by a covariance matrix, which describes how uncertainty spreads across different directions.

Standard information-theoretic tools often fail here because they require "covariance proportionality"—essentially assuming the noise has the same shape in every stage of a problem. As Tie Liu 0002 noted in An Extremal Inequality Motivated by Multiterminal Information-Theoretic Problems, this requirement is rarely met in multiterminal problems like vector Gaussian broadcast channels. To solve this, researchers have developed new extremal inequalities that replace scalar parameters with positive semidefinite matrices, providing a more flexible framework for multi-antenna systems, as demonstrated by Liu et al. in their vector generalization of Costa’s EPI.

Fisher Information as a Bridge

To prove that Gaussian signaling is optimal in these high-dimensional spaces, researchers often turn to the de Bruijn identity, which links differential entropy to Fisher information. The intuition is that if you can show a distribution minimizes Fisher information, you can bound its entropy.

Tamer Başar established a foundational piece of this puzzle in Optimum Fisherian Information for Multivariate Distributions, proving that the Gaussian distribution is indeed the "least informative" noise structure in multivariate settings. Building on this, Ugur et al. used Fisher information to provide the first complete characterization of the Vector Gaussian CEO Problem Under Logarithmic Loss. By shifting the metric to logarithmic loss, the authors were able to use the properties of Fisher information to prove that simple Gaussian signaling remains optimal even when multiple sensors observe a vector source.

Closing the Gap in Successive Refinement

One of the most challenging areas for vector sources is successive refinement—the process of sending a coarse description of data followed by details that improve accuracy. In scalar cases, we know how to do this optimally. In vector cases, the "water-filling" strategy (allocating bits to the most important dimensions) becomes much more complex because the dimensions interact.

To tackle this, Xu et al. introduced a "monotone path" argument combined with a "doubling trick" in Vector Gaussian Successive Refinement With Degraded Side Information. This approach allows for a rigorous proof of Gaussian optimality by creating a continuous path between different distortion constraints, effectively "smoothing" the transition between dimensions that would otherwise be mathematically intractable.

Beyond Distortion: Perception and Realism

Recent work has expanded these vector constraints beyond simple reconstruction accuracy (distortion) to include perception (how "real" the reconstructed signal looks). In Rate-Distortion-Perception Tradeoff for Gaussian Vector Sources, Qian et al. show that vector sources behave fundamentally differently than scalar ones under perceptual constraints. While traditional compression might discard low-variance components, maintaining the "statistical look" of a vector requires allocating bits to every component. This leads to a "generalized water-filling" solution, where the geometry of the entire vector space must be preserved to satisfy the receiver's perceptual requirements.

Algorithmic Proofs for Higher Dimensions

As the mathematical complexity of these multivariate derivatives grows, some researchers have moved from manual proofs to computational ones. Guo et al. demonstrated in Lower Bound for Derivatives of Costa's Differential Entropy that information-theoretic conjectures can be transformed into optimization problems. By using Semidefinite Programming (SDP) and Sum-of-Squares (SOS) polynomials, the authors were able to prove Gaussian optimality for higher-order derivatives of entropy that had previously resisted univariate-style proofs. This suggests that the future of vector information theory may lie in bridging classical analysis with modern convex optimization.

Key Foundational Work

Optimum Fisherian Information for Multivariate Distributions (1977) — contributed 1 concept elements to this direction.
An Extremal Inequality Motivated by Multiterminal Information-Theoretic Problems (2007) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_scalar_entropy_power_constraints_142535a9,
  title = {scalar entropy-power constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scalar-entropy-power-constraints/rdr_142535a9f386455592faa34c267a182c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_scalar_entropy_power_constraints_142535a9_v1,
  title = {scalar entropy-power constraints},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scalar-entropy-power-constraints/rdr_142535a9f386455592faa34c267a182c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake