Introduction
In classical control theory, decision-makers (DMs) are typically assumed to have either perfect information or access to data through fixed, pre-defined sensors. However, in biological and economic systems, agents often face internal processing limits, forcing them to choose what to prioritize. This concept, known as Rational Inattention, suggests that agents optimize their observation channels subject to a strict information budget (measured in bits or nats).
This paper provides a rigorous mathematical framework for rationally inattentive control of Markov processes. By bridging Information Theory and Control Theory, the authors demonstrate how to optimize systems when the DM must design their own "sensor" under Shannon-style constraints.
Problem Formulation
The core challenge lies in the joint optimization of the observation channel and the control policy. Unlike standard problems where the sensor is fixed, here the DM selects the mapping from states to observations to minimize cost while staying within a mutual information budget.
As shown in the figure above, the system consists of a controlled process $X_t$, an observation channel that produces $Z_t$, and a controller that generates actions $U_t$. The goal is to optimize the entire loop, accounting for the "cost" of information processing.
Methods
The authors focus on average-cost optimal control, employing several key technical strategies:
- Policy Simplification: They prove that choosing an observation channel and a controller separately is equivalent to selecting a single randomized stationary policy mapping states to actions, provided the policy's mutual information is penalized.
- Convex Analytic Approach: By reformulating the problem as linear programming over probability measures, the authors transform a traditionally non-convex problem into a tractable convex optimization task.
- Information-Constrained Bellman Equation (IC-BE): A central contribution is the derivation of the IC-BE. This equation generalizes the standard Bellman equation by replacing the
minoperator with a variational "soft-min" form that incorporates the information penalty. - Rate-Distortion Theory: The framework utilizes the Shannon Distortion-Rate Function (DRF), where control cost serves as the "distortion" and the information budget as the "rate."
Key Results and Insights
The research yields several significant findings:
- Existence and Convergence: The authors provide formal proofs for the existence of optimal information-constrained control laws and establish conditions for the convergence of mutual information to a steady-state budget.
- Information-Constrained LQG: In the scalar Linear-Quadratic-Gaussian (LQG) case, the authors derived a modified Riccati equation (IC-DARE). Notably, they found that the "certainty equivalence" principle—the separation of estimation and control—breaks down at low information rates, necessitating joint optimization.
- Performance Gains: Numerical simulations indicate that the proposed information-constrained controllers outperform standard "certainty-equivalent" designs when the information budget is limited.
- Convexity and Representation: A key insight is that the problem simplifies significantly if the DM is free to choose the representation of the data (the observation space), allowing for a reduction to an infinite-dimensional convex program.
Conclusion and Future Work
While this work provides a solid theoretical foundation, several avenues for future research remain. The current framework assumes "memoryless" observation channels, whereas real-world inattention often involves internal states and memory. Additionally, while the optimization is convex, solving it for high-dimensional systems remains computationally demanding. Future work will likely focus on extending these results to vector systems and exploring approximate dynamic programming techniques to handle larger state spaces.
Summary updated