Value of Information in Minimum-Rate LQG Control

Touraj Soleymani, Sandra Hirche, John S. BarasIFAC-PapersOnLine 2017 · DOI

TL;DR

This paper tackles the challenge of controlling mobile cyber-physical systems with limited communication bandwidth. They introduce a novel framework for LQG control over unreliable networks, proving that optimal control and sampling policies can be achieved by focusing on the "value of information" – essentially, how much a piece of data is worth for improving control performance.

What Problem Does This Paper Address?

Mobile cyber-physical systems (CPS) often face a critical bottleneck: communication. Devices (like sensors on a robot) have limited power, leading to asymmetrical communication capabilities. The "uplink" channel, sending data from the device to a base station, is often constrained in bandwidth and prone to packet loss. This makes traditional feedback control difficult, as the base station might not receive timely or complete information about the system's state. Existing information-theoretic measures (like mutual information) quantify data quantity but don't capture its usefulness for a specific task like control. This paper aims to bridge this gap by developing a control framework that explicitly considers the value of information for optimal decision-making under communication constraints.

Methods

The authors employ a combination of dynamic programming and information theory to solve a partially observed Linear Quadratic Gaussian (LQG) control problem. Key methodological contributions include:

  • Packet-Deletion Channel Model: They model the uplink communication as a zero-delay packet-deletion channel, meaning packets are either received or lost without notification. This is a simplification that reduces communication overhead and simplifies the filter design.
  • Event-Driven Sampling: Instead of sampling data at fixed intervals (uniform sampling), they propose an event-driven sampling policy. This means data is only sent when its "value of information" exceeds a certain threshold, making communication more efficient.
  • Value of Information (VoI): They define and utilize a novel measure of information – the "value of information" – which quantifies the improvement in future system performance gained from receiving specific causal information.
  • Separation Principle: They prove that, under their framework, the separation principle between control and estimation holds. This is a significant result, as it means the optimal control policy can be derived independently from the optimal estimation policy.
  • Optimal Policies: They characterize the optimal control policy as a "certainty equivalent" policy (acting as if the estimated state is the true state) and the optimal sampling policy as a "threshold" policy based on the VoI.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and simulation-based validation is not presented.

Key Insights and Ideas

  • Information is not just quantity, but quality for a task: The paper strongly emphasizes that for control problems, the usefulness of information (its value) is more critical than just its raw amount. This leads to more efficient communication strategies.
  • Event-driven sampling is key to efficiency: By only transmitting data when it's deemed valuable, systems can significantly reduce communication load while maintaining or even improving control performance.
  • The "Value of Information" is a powerful concept: Defining and quantifying the VoI allows for a principled approach to designing optimal sampling strategies. It directly links information acquisition to control objectives.
  • Separation Principle Holds (under specific conditions): The proof that the separation principle holds in this context is a major theoretical contribution. It simplifies the design process by allowing independent optimization of the controller and the estimator.
  • VoI is quadratic in innovation: A surprising and elegant result is that the value of information can be expressed as a quadratic function of the "innovation" (the difference between the actual measurement and the predicted measurement). This provides a concrete mathematical form for the VoI.

Key Results

  • The paper establishes a framework for minimum-rate LQG control over packet-deletion channels with event-driven sampling.
  • It proves that the optimal control policy is a certainty equivalent policy.
  • It demonstrates that the optimal sampling policy is a threshold policy, where data is sampled if its value of information exceeds a threshold.
  • A key theoretical result is the proof that the value of information is a quadratic function of the innovation.
  • The paper shows that the separation principle between control and estimation holds in this setting.

Limitations and Future Work

  • Ideal Downlink Channel: The paper assumes an ideal feedback channel from the base station to the node. In real-world scenarios, this channel might also have limitations.
  • Zero-Delay Packet-Deletion: The packet-deletion channel is assumed to have zero delay. Incorporating communication delays would add another layer of complexity.
  • High-Resolution Quantization: The analysis relies on high-resolution quantization, which might not always be feasible.
  • Offline Tuning of VoI: While the paper provides the parametric architecture for the estimation cost-to-go (related to VoI), it suggests that tuning these parameters might require techniques from approximate dynamic programming, implying an offline or computationally intensive process.

Future work could explore relaxing these assumptions, such as investigating scenarios with noisy or delayed feedback channels, finite-resolution quantization, and developing online or adaptive methods for determining the value of information and sampling thresholds. Extending this framework to more complex system dynamics and communication models is also a promising direction.

Summary updated

Abstract

This study concerns fundamental limitations in control of mobile cyber-physical systems. In these systems, communication between a node and its base station due to limited power of the node is asymmetrical in terms of bandwidth and signal-to-noise ratio. The framework we develop in this paper is for partially observed linear quadratic Gaussian (LQG) control over communication networks in which the forward channel transporting measurements is modeled by a zero-delay packet-deletion channel and the feedback channel transporting control inputs is assumed ideal. Making use of dynamic programming, we characterize the optimal control and the optimal sampling policies that achieve the minimum data rate required for a guaranteed level of control performance. In particular, we prove that in the presence of event-driven sampling the adopted filter is optimal and the separation principle between control and estimation holds. We show that the optimal control policy is a certainty equivalent policy and the optimal sampling policy is a threshold policy expressed in terms of the value of information. Furthermore, we prove that the value of information is a quadratic function of the innovation.
BibTeX
@inproceedings{soleymani2017value,
  author = {Touraj Soleymani and Sandra Hirche and John S. Baras},
  title = {{Value of Information in Minimum-Rate LQG Control}},
  booktitle = {IFAC-PapersOnLine},
  year = {2017},
  doi = {10.1016/j.ifacol.2017.08.1322},
}
Figure