Robust Distributed Source Coding

Jun Chen, Toby Berger

Abstract

We consider a distributed source coding system in which several observations are communicated to the decoder using limited transmission rate. The observations must be separately coded. We introduce a robust distributed coding scheme which flexibly trades off between system robustness and compression efficiency. The optimality of this coding scheme is proved for various special cases.

Introduction

In distributed source coding, such as in sensor networks, multiple encoders observe correlated data and transmit it to a central decoder. Traditional research focuses on compression efficiency—minimizing the total bit rate by exploiting correlations. However, these systems are often brittle; if one encoder fails, the joint reconstruction at the decoder may fail entirely. Conversely, the "multiple description" problem focuses on robustness, ensuring signal recovery from any subset of packets, but typically assumes a centralized encoder.

This paper bridges this gap by introducing Robust Distributed Source Coding. This framework addresses scenarios where encoders are physically separate yet the system must remain resilient to individual link failures while maintaining high efficiency.

System Model and Methodology

The authors propose a unified coding scheme that integrates random binning (for exploiting correlation) and multiple description coding (for redundancy). The system model involves a source $X$ observed as $Y_1$ and $Y_2$ by two separate encoders.

The system model for robust distributed source coding, showing separate encoders and multiple decoders for individual and joint reconstruction.

As shown in Figure 1, the encoders send rates $R_1$ and $R_2$. The system includes three decoders:

  • Decoder 1 and 2: Provide individual reconstructions $\hat{X}_1$ and $\hat{X}_2$ with distortions $D_1$ and $D_2$, ensuring robustness if one link fails.
  • Decoder 3: Provides a joint reconstruction $\hat{X}_3$ with distortion $D_3$, maximizing efficiency when both links are active.

The core methodology introduces auxiliary random variables: $U_i$ for "coarse" information (decodable individually) and $W_i$ for "fine" details (requiring joint decoding). By leveraging the "Common Information" between sensors, the scheme can transition between distributed and centralized coding regimes.

Key Results and Discussion

The paper evaluates the proposed scheme using the Quadratic Gaussian case, where observations are corrupted by independent Gaussian noise. A primary finding is the fundamental trade-off between robustness and efficiency: increasing resilience to sensor failure necessitates sending redundant information that does not contribute to joint compression gains.

Comparison of the achievable rate region for the proposed scheme versus the IPPR scheme, demonstrating the improved efficiency of the new framework.

The authors define the achievable region for the quintuple $(R_1, R_2, D_1, D_2, D_3)$. As illustrated in Figure 2, the proposed framework outperforms existing methods like the IPPR (Ishwar-Puri-Pradhan-Ramchandran) scheme. The IPPR scheme is shown to be a special case that falls short of the optimal rate-distortion boundary established in this work. Furthermore, the authors prove optimality for several special cases, such as when one encoder has an infinite bit rate.

Conclusion

This work provides a robust framework for distributed source coding that balances the needs of efficiency and resilience. While the complete characterization of the general Gaussian rate-distortion region remains an open challenge, this unified approach offers a significant step forward. Future work will focus on reducing the implementation complexity of multi-layer coding and exploring the convergence of distributed regions to centralized ones as sensor noise approaches zero.

Summary updated

Abstract

We consider a distributed source coding system in which several observations are communicated to the decoder using limited transmission rate. The observations must be separately coded. We introduce a robust distributed coding scheme which flexibly trades off between system robustness and compression efficiency. The optimality of this coding scheme is proved for various special cases.
BibTeX
@article{chen2008robust,
  author = {Jun Chen and Toby Berger},
  title = {{Robust Distributed Source Coding}},
  journal = {IEEE Trans. Information Theory},
  year = {2008},
  doi = {10.1109/tit.2008.926389},
}

Loading related content...

Figure