Abstract: We consider the problem of information embedding where the encoder modifies a white Gaussian host signal in a power-constrained manner to encode a message, and the decoder recovers both the embedded message and the modified host signal. This partially extends the recent work of Sumszyk and Steinberg to the continuous-alphabet Gaussian setting. Through a control-theoretic lens, we observe that the problem is a minimalist example of what is called the "triple role" of control actions. We show that a dirty-paper-coding strategy achieves the optimal rate for perfect recovery of the modified host and the message for any message rate. For imperfect recovery of the modified host, by deriving bounds on the minimum mean-square error (MMSE) in recovering the modified host signal, we show that DPC-based strategies are guaranteed to attain within a uniform constant factor of 16 of the optimal weighted sum of power required in host signal modification and the MMSE in the modified host signal reconstruction for all weights and all message rates. When specialized to the zero-rate case, our results provide the tightest known lower bounds on the asymptotic costs for the vector version of a famous open problem in decentralized control: the Witsenhausen counterexample. Numerically, this tighter bound helps us characterize the asymptotically optimal costs for the vector Witsenhausen problem to within a factor of 1.3 for all problem parameters, improving on the earlier best known bound of 2.
Introduction
In traditional information embedding, such as digital watermarking, an encoder hides a message within a "host" signal. The primary objective is typically to recover the message while ensuring the host signal remains relatively undistorted. However, in decentralized control systems, the host signal often represents the state of a system. An agent modifies this state to achieve a specific control goal, but subsequent agents must accurately estimate this modified state to perform their own tasks correctly.
This paper bridges the gap between information theory and decentralized control by analyzing a Gaussian information embedding problem where the decoder must recover both an embedded message and the modified host signal. This setup highlights the complex interplay between communication and control.
System Model and the Triple Role of Control
The core of the study is a system where a host signal $X_0^m$ is modified by an encoder $\mathcal{E}$ to produce a modified host signal $X_1^m$. This modification incorporates an external message $M$ that needs to be communicated. The receiver observes a noisy version of the modified host, $Y^m$, and must decode both the message $\hat{M}$ and an estimate of the modified host $\hat{X}_1^m$.
This framework exemplifies the "triple role of control," where a single action must simultaneously:
- Minimize immediate cost: Use low power ($V^m$) to modify the signal.
- Improve estimability: Shape the signal to make it easier for the receiver to estimate the modified state.
- Communicate information: Embed an explicit message to coordinate future actions.
Methodology: Dirty-Paper Coding and MMSE Bounds
The authors employ a combination of information-theoretic proofs and numerical analysis to find the optimal tradeoff between modification power and reconstruction error (MMSE).
Achievability via DPC
The proposed strategy utilizes Dirty-Paper Coding (DPC). While DPC is traditionally used to cancel known interference when sending a message, the authors adapt it here for signal shaping. By having the decoder recover the auxiliary random variable used in DPC, it can better reconstruct the modified host signal itself, even when the explicit message rate is zero.
Converse and Lower Bounds
New lower bounds on the MMSE are derived using the data-processing inequality and Fano’s inequality. The analysis focuses on how the correlation between the original host and the control signal affects the receiver's ability to estimate the final state. This theoretical grounding allows for a rigorous comparison between proposed strategies and the absolute physical limits of the system.
Connection to the Witsenhausen Counterexample
A significant contribution of this work is its application to the Witsenhausen counterexample, a famous 40-year-old open problem in decentralized control. The counterexample demonstrates that linear control is sub-optimal in certain decentralized settings, but the optimal non-linear strategy remains unknown.
By viewing Witsenhausen's problem as a zero-rate information embedding problem (as shown in Figure 2b), the authors apply their DPC-based bounds to the vector version of the problem (where signal length $m \to \infty$). They demonstrate that the tension between signaling and control is fundamentally the same challenge addressed in their information embedding framework.
Key Results
- Constant Factor Optimality: The authors prove that DPC-based strategies stay within a factor of 16 of the optimal cost for all possible message rates and system parameters.
- Witsenhausen Breakthrough: For the vector Witsenhausen problem, they improved the known gap between the best strategy and the theoretical limit from a factor of 2 down to 1.3.
- Non-linear Superiority: The results reinforce that in decentralized settings, "certainty equivalence" fails. The encoder must actively "help" the decoder by choosing non-linear strategies like DPC to shape the state for better estimability.
Limitations and Future Work
The findings primarily apply to the asymptotic "vector" case. The scalar case (the original Witsenhausen problem) remains significantly more difficult to bound tightly. Additionally, there is a lack of tightness in the "high distortion" regime where the power budget for modification is very low. Future research may explore more complex channel models where the noise might be correlated with the original host signal.
Summary updated