Introduction
In communication theory, "colored" noise refers to noise that is correlated over time, a more realistic model than standard "white" noise. While the capacity of Gaussian channels with white noise is well-understood, determining the capacity with feedback—where the transmitter observes previous channel outputs—and colored noise has remained a significant challenge for decades.
The seminal work by Cover and Pombra (1989) established a mathematical limit for this capacity. However, their formulation was not "computable" in a practical sense, as it required solving an optimization problem where dimensions grew to infinity. This research bridges that gap by providing a computable, finite-dimensional formula for Multiple-Input Multiple-Output (MIMO) channels where noise is modeled via linear state-space systems.
Methodology: Bridging Control and Information Theory
The authors develop a framework that integrates information theory with classical control theory and convex optimization:
- State-Space Modeling: Colored noise is represented as the output of a time-invariant state-space system, allowing the channel to be analyzed in an "observer form" using Kalman filtering principles.
- Sequential Convex Optimization (SCOP): The $n$-block capacity is formulated as a sequence of optimizations. The authors demonstrate that these constraints possess a recursive structure that remains tractable as $n$ increases.
- Linear Matrix Inequalities (LMIs): The central contribution is the transformation of the capacity problem into a maximal determinant optimization problem subject to LMIs. This class of problems is convex and can be efficiently solved using standard numerical tools like CVX.
Key Insights
- The Riccati Connection: A profound link is established between feedback capacity and the Riccati equation, a fundamental tool in optimal control. The LMI constraints in the proposed optimization are essentially tight relaxations of these equations.
- Optimality of Time-Invariance: The paper proves that simple time-invariant signaling strategies are optimal, even when the underlying noise process is non-stationary. This significantly simplifies practical transmitter design.
- Information as Smoothing: Feedback capacity is shown to be related to a "smoothing" problem—specifically, how much the receiver can reduce uncertainty about the initial noise state given the sequence of channel outputs.
Numerical Results and Performance Gains
The proposed framework allows for the direct computation of capacity for various noise processes, such as Auto-Regressive (AR) and Moving-Average (MA) models.
As shown in Figure 1, the "feedback gain"—the difference between capacity with and without feedback—becomes substantial as the noise correlation (represented by the regression parameter $\beta$) increases. For highly correlated AR noise, the feedback capacity significantly outperforms the standard water-filling (non-feedback) approach. Furthermore, the authors demonstrate that for scalar channels, their coding scheme achieves an error probability that decays "doubly-exponentially" with block length, a marked improvement over standard exponential decay.
Conclusion and Future Work
This work provides the definitive computable formula for MIMO feedback capacity (Theorem 1). While it offers a numerical path for any ARMA process, some challenges remain. The explicit coding algorithms are currently most detailed for scalar channels; extending these to a full spectral analysis for MIMO systems is a subject for future research. Additionally, while the Gaussian assumption is central to this LMI-based approach, extending these techniques to non-Gaussian or discrete-alphabet channels remains an open frontier.
Summary updated