Abstract
We study the convexity of mutual information as a function of time along the flow of the Ornstein-Uhlenbeck process. We prove that if the initial distribution is strongly log-concave, then mutual information is eventually convex, i.e., convex for all large time. In particular, if the initial distribution is sufficiently strongly log-concave compared to the target Gaussian measure, then mutual information is always a convex function of time. We also prove that if the initial distribution is either bounded or has finite fourth moment and Fisher information, then mutual information is eventually convex. Finally, we provide counterexamples to show that mutual information can be nonconvex at small time.
Introduction
In information theory and statistical physics, understanding how information evolves as a system relaxes toward equilibrium is a fundamental challenge. The Ornstein-Uhlenbeck (OU) process serves as a primary model for this relaxation, describing a random walk (Brownian motion) subject to a linear restorative force that pulls the distribution toward a central Gaussian target. While the behavior of entropy along this flow is well-characterized—it increases in a convex manner—the evolution of mutual information $I(X_0; X_t)$ has remained more elusive. This paper investigates the convexity of mutual information as a function of time $t$, identifying conditions under which it eventually becomes convex and demonstrating cases where convexity fails in the early stages of the process.
Theoretical Framework and Methodology
The authors approach the problem by treating the OU process as a gradient flow in the space of probability measures, specifically the Wasserstein space. To analyze the second derivative of mutual information with respect to time, they derive novel identities relating it to "relative second-order Fisher information" ($K_\nu$).
A central concept in their analysis is $\lambda$-strong log-concavity. A distribution is $\lambda$-strongly log-concave if its density $f$ satisfies $\nabla^2 \log f \le -\lambda I$. This property measures how "peaked" or "curved" a distribution is, and the authors track how this curvature evolves as the OU process smooths the distribution over time. By combining stochastic calculus with functional analysis, they establish rigorous bounds on the time required for mutual information to transition into a convex regime.
Key Findings
The "Eventually Convex" Rule
The research demonstrates that for a broad class of initial distributions—including those with bounded support, strong log-concavity, or finite fourth moments and Fisher information—mutual information $I(X_0; X_t)$ will inevitably become a convex function of time. This suggests that after a certain "time threshold," the diffusive noise of the OU process dominates the initial structural complexities of the distribution, leading to a predictable, convex decay of information.
Log-Concavity Thresholds
Theorem 1 provides a specific condition for global convexity: if the initial distribution is $\lambda$-strongly log-concave and $\lambda \ge \alpha/2$ (where $\alpha$ represents the OU drift coefficient), then mutual information is convex for all $t \ge 0$. For distributions that are less strongly log-concave, convexity is guaranteed only after a time $t(\lambda)$.
Small-Time Non-Convexity
Contrary to entropy, mutual information is not universally convex. At very small time scales, the process can exhibit concave behavior. This is particularly evident in distributions that are highly structured or discrete. For instance, if the initial state consists of two distinct point masses, the early phase of the OU process involves the "blurring" of these points into small Gaussian clouds. During this initial expansion, the mutual information remains relatively flat, creating a plateau that violates convexity.
Numerical Evidence
To validate their theoretical results, the authors provide numerical simulations of mutual information for specific "toy" distributions. These examples highlight the transition from non-convexity to convexity.
As seen in the figure, for both a mixture of point masses and a mixture of Gaussians, the mutual information curve exhibits a characteristic "dip" or plateau at small $t$ before settling into a smooth, convex decay. In the case of point masses (a), the curve starts at $\log(2)$ (the discrete entropy) and remains nearly constant while the points begin to diffuse, before eventually following the predicted convex path toward zero.
Conclusion and Future Directions
This work establishes that eventual convexity is a robust property of mutual information under the Ornstein-Uhlenbeck flow. While the initial stages of diffusion can be complex and non-convex, the long-term behavior is governed by the smoothing properties of the Gaussian target.
Future research may focus on tightening the calculated time thresholds for convexity, as current proofs likely overestimate the time required for the transition. Additionally, extending these results to general Fokker-Planck processes—where the target equilibrium may not be Gaussian—remains an open and significant question in the field of information-theoretic functional inequalities.
Summary updated