Abstract
Determining the feedback capacity of Gaussian channels with memory has long been a fundamental challenge in information theory. This paper provides a unified, computable solution for Multiple-Input Multiple-Output (MIMO) Gaussian channels by modeling them within a general linear state-space framework. By utilizing directed information and Kalman filtering recursions, the authors transform the capacity problem into a tractable convex optimization problem subject to Linear Matrix Inequality (LMI) constraints. This framework not only subsumes existing models for colored noise and intersymbol interference but also provides a definitive tool for analyzing the impact of feedback delays in complex communication systems.
Introduction
In communication theory, the concept of "capacity" represents the theoretical upper bound on the rate at which information can be transmitted reliably over a channel. While the capacity of memoryless Gaussian channels is well-understood, real-world channels often exhibit "memory," where current outputs are influenced by previous inputs or noise. Calculating capacity becomes significantly more complex when the sender has access to real-time feedback from the receiver, a scenario common in many modern control and communication applications.
Historically, solutions for feedback capacity were fragmented, often restricted to specific noise models like colored Gaussian noise or specific structures like intersymbol interference (ISI). This paper addresses the lack of a unified theory by introducing a framework that characterizes the feedback capacity for a broad class of MIMO Gaussian channels with memory.
Methodology: The State-Space Framework
The core innovation of this work lies in the application of a linear state-space model to describe the channel. In this model, a hidden state vector captures the system's memory, evolving over time based on previous states, current inputs, and Gaussian noise. This approach allows for a generalized representation that encompasses various previously studied channel models.
Directed Information and Kalman Filtering
To characterize capacity in the presence of feedback, the authors utilize directed information rather than standard mutual information. Directed information is the mathematically rigorous metric for systems where the input can depend on past outputs. To manage the evolution of the system state, the authors employ Kalman filter recursions. These recursions track the "encoder's estimate" and the "decoder's estimate" of the hidden channel state, providing the necessary statistics to formulate the capacity problem.
Convex Optimization and LMIs
The paper’s primary technical contribution is the proof that the feedback capacity is equivalent to the maximum of a concave function subject to Linear Matrix Inequality (LMI) constraints. By framing the problem this way, the authors move the calculation of capacity from an abstract theoretical exercise to a computable engineering task. Researchers can now solve for capacity using standard numerical optimization tools, such as CVX, rather than relying on exhaustive searches or complex analytical derivations for every new channel configuration.
Key Insights and Results
Unification of Gaussian Models
The state-space approach demonstrates that diverse problems—including colored noise, ISI, and output-dependent noise—are special cases of a single general equation. This unification allows the framework to recover and validate results from landmark papers in the field, providing a consistent theoretical foundation for Gaussian feedback capacity.
The Impact of Feedback Delay
A critical finding of this research is the quantitative analysis of feedback delay. The authors demonstrate that even a single time-step of delay in the feedback loop can significantly reduce the achievable capacity. In real-world systems where instantaneous feedback is physically impossible due to propagation or processing times, this insight is vital for realistic performance modeling. Through numerical simulations of auto-regressive (AR) Gaussian noise models, the authors show that while the gap between instantaneous and delayed feedback narrows as noise memory increases, the capacity loss in stationary regimes remains substantial.
The Detectability Condition
The solution relies on a "detectability" condition—a standard concept in control theory—to ensure system stability and optimization convergence. While the authors prove this condition is always satisfied for scalar (SISO) models, they conjecture it also holds for the general MIMO case. This "Detectability Conjecture" suggests that the proposed convex optimization remains robust across all optimal MIMO solutions.
Conclusion
This work bridges the gap between information theory and control theory, providing a computable and unified solution for the feedback capacity of MIMO Gaussian channels with memory. By transforming the problem into a convex optimization framework, it enables practical calculation of limits for complex communication systems.
Future research directions include extending these linear Gaussian results to non-linear or non-Gaussian regimes, which remain significant challenges in the field. Additionally, proving the detectability conjecture for the general MIMO case would further solidify the theoretical requirements of the state-space framework. For now, this methodology stands as a definitive tool for understanding the fundamental limits of communication with feedback and memory.
Summary updated