Minimum-Information LQG Control - Part II: Retentive Controllers

Roy Fox, Naftali Tishby

Abstract

Retentive (memory-utilizing) sensing-acting agents may operate under limitations on the communication between their sensing, memory and acting components, requiring them to trade off the external cost that they incur with the capacity of their communication channels. In this paper we formulate this problem as a sequential rate-distortion problem of minimizing the rate of information required for the controller's operation under a constraint on its external cost. We reduce this bounded retentive control problem to the memoryless one, studied in Part I of this work, by viewing the memory reader as one more sensor and the memory writer as one more actuator. We further investigate the structure of the resulting optimal solution and demonstrate its interesting phenomenology.

Introduction

In classical control theory, such as the standard Linear-Quadratic-Gaussian (LQG) model, controllers are typically assumed to possess infinite bandwidth. They are modeled as having perfect sensing, instantaneous processing, and infinite memory. However, in physical systems ranging from biological brains to remote-controlled drones, communication and memory are finite resources. This paper addresses the design of optimal "retentive" controllers—those that maintain an internal state to track the world over time—under strict informational constraints.

Problem Formulation and Methodology

The authors utilize a Sequential Rate-Distortion framework to minimize the Shannon information rate required for controller operation while maintaining control costs below a specified threshold. The central innovation is a mathematical reduction: a retentive controller is modeled as a memoryless controller operating on an augmented system.

Block diagram of a retentive control system where memory acts as a feedback loop with finite capacity.

In this augmented view:

  • The memory reader (accessing past states) is treated as an additional sensor.
  • The memory writer (storing current states for future use) is treated as an additional actuator.

This reduction allows the application of iterative algorithms involving forward updates of system distributions, backward updates of control policies, and Eigenvalue Decomposition (EVD) to identify the optimal "modes" of the system to prioritize within the bit budget.

Experimental Validation

The theory is validated using a simulated double mass-spring-damper system. This 4-dimensional physical system (comprising two positions and two velocities) provides a complex, coupled environment to test how memory compensates for limited sensing. The system is both controllable and observable, making it an ideal benchmark for evaluating the trade-off between information rate and control performance.

Key Results and Phenomenology

The research yields several significant insights into the nature of information-constrained control:

  1. Memory as a Communication Channel: Memory is effectively a channel from the "past self" to the "future self," allowing information-theoretic tools to quantify the cost of remembering.
  2. MMSE Canonical Form: The optimal controller only needs to store the Minimum Mean Square Error (MMSE) estimate of the state, rather than raw data, to act optimally in the future.
  3. Phase Transitions: As the information budget ($\beta$) increases, the controller's performance does not improve linearly. Instead, it undergoes discrete "phase transitions" where the controller suddenly gains enough capacity to manage an additional dimension or "mode" of the physical system.
Cost vs. information rate (log2 β) for memoryless and retentive controllers, illustrating phase transitions (d=0 to d=4) and the performance advantage of retention.

As shown in the results, retentive controllers significantly outperform memoryless ones. For a given bit budget, a retentive controller achieves lower costs by integrating information over time. In the 4D mass-spring system, the retentive controller eventually "unlocks" all four modes, whereas the memoryless controller remains limited by the rank of its sensor/actuator matrices, regardless of the bit budget.

Conclusion

By treating memory as a finite communication resource, this work provides a unified framework for understanding the informational limits of control. While currently focused on Linear-Gaussian dynamics, the discovery of phase transitions suggests a rich underlying structure in how agents allocate internal resources. Future work may extend these principles to non-linear systems and explore architectures with separate capacity constraints for sensing and memory.

Summary updated

Abstract

Retentive (memory-utilizing) sensing-acting agents may operate under limitations on the communication between their sensing, memory and acting components, requiring them to trade off the external cost that they incur with the capacity of their communication channels. In this paper we formulate this problem as a sequential rate-distortion problem of minimizing the rate of information required for the controller's operation under a constraint on its external cost. We reduce this bounded retentive control problem to the memoryless one, studied in Part I of this work, by viewing the memory reader as one more sensor and the memory writer as one more actuator. We further investigate the structure of the resulting optimal solution and demonstrate its interesting phenomenology.
BibTeX
@inproceedings{fox2016minimum,
  author = {Roy Fox and Naftali Tishby},
  title = {{Minimum-Information LQG Control - Part II: Retentive Controllers}},
  booktitle = {CDC},
  year = {2016},
  url = {https://openreview.net/forum?id=7kNWGAlLJw},
}
Figure