Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip

Yuxuan Song, Minkai Xu, Lantao Yu, Hao Zhou, Shuo Shao, Yong Yu

Abstract

Traditional communication systems rely on the separation of source and channel coding, which is often sub-optimal for finite-length data blocks in real-world applications. This paper presents Infomax Adversarial-Bit Flip (IABF), a joint source-channel coding (JSCC) framework that leverages mutual information maximization and adversarial regularization. By maximizing the information content of binary codewords and training against worst-case bit-flip scenarios, IABF achieves superior robustness and reconstruction accuracy compared to existing neural JSCC methods.

Introduction

In classical information theory, Shannon’s separation theorem posits that source coding (compression) and channel coding (error correction) can be treated independently without loss of optimality. However, this optimality holds only in the limit of infinite block lengths. For modern low-latency applications like IoT or real-time streaming, finite-length constraints make joint source-channel coding (JSCC) more effective.

Recent neural JSCC approaches, such as NECST, utilize autoencoders to learn end-to-end mappings. Despite their promise, these models often lack robustness to channel noise and fail to produce highly informative codewords. The IABF framework addresses these limitations by explicitly optimizing for informativeness and worst-case error resilience.

Proposed Method: IABF

The IABF architecture introduces two primary innovations to the standard autoencoder-based JSCC pipeline:

  1. Infomax Encoder: To ensure the binary codeword $y$ captures maximal information about the input $x$, the model maximizes the Mutual Information (MI) between them. Since direct MI computation is intractable for high-dimensional bit-strings, IABF employs a Total Correlation (TC) penalty. A discriminator network is trained to distinguish between the joint distribution of bits and the product of their marginals, encouraging the encoder to produce informative, independent bits.
  2. Adversarial Bit Flip Regularization: To enhance decoder robustness, IABF moves beyond random noise injection. It identifies "vulnerable" bits by calculating the gradient of the reconstruction loss with respect to the codeword. By "virtually" flipping these critical bits during training, the decoder learns to reconstruct data even under adversarial noise conditions.
Figure 1: The IABF architecture combining Mutual Information (MI) loss for the encoder and gradient-based adversarial bit flipping for decoder regularization.

Experimental Evaluation

The performance of IABF was evaluated on several image benchmarks, including BinaryMNIST, MNIST, Omniglot, and CIFAR10. These datasets represent varying levels of complexity for testing reconstruction quality across different noise regimes.

Key Results

Experimental results demonstrate that IABF consistently outperforms state-of-the-art baselines like NECST:

  • Reconstruction Accuracy: IABF achieved lower $L^2$ reconstruction errors across all tested noise levels (ranging from 10% to 40% bit-flip probability).
  • Training Stability: Unlike previous models that exhibit "spiky" training curves and susceptibility to overfitting, IABF maintains a stable learning trajectory throughout the training process.
  • Information Distribution: The adversarial mechanism forces the encoder to distribute information uniformly across the codeword, preventing the model from over-relying on a few "fragile" bits that would cause catastrophic failure if flipped.
Figure 4: Comparison of reconstruction error over training epochs at a 0.3 noise level. IABF shows significantly better stability and lower error than the NECST baseline.

Conclusion

IABF represents a significant step forward in neural joint source-channel coding. By integrating information-theoretic objectives with adversarial training, it provides a robust solution for data transmission over noisy channels. Future work includes extending the adversarial bit-flip concept to non-binary channels and exploring active learning strategies to further optimize the training process against complex noise patterns.

Summary updated

Abstract

Although Shannon theory states that it is asymptotically optimal to separate the source and channel coding as two independent processes, in many practical communication scenarios this decomposition is limited by the finite bit-length and computational power for decoding. Recently, neural joint source-channel coding (NECST) is proposed to sidestep this problem. While it leverages the advancements of amortized inference and deep learning to improve the encoding and decoding process, it still cannot always achieve compelling results in terms of compression and error correction performance due to the limited robustness of its learned coding networks. In this paper, motivated by the inherent connections between neural joint source-channel coding and discrete representation learning, we propose a novel regularization method called Infomax Adversarial-Bit-Flip (IABF) to improve the stability and robustness of the neural joint source-channel coding scheme. More specifically, on the encoder side, we propose to explicitly maximize the mutual information between the codeword and data; while on the decoder side, the amortized reconstruction is regularized within an adversarial framework. Extensive experiments conducted on various real-world datasets evidence that our IABF can achieve state-of-the-art performances on both compression and error correction benchmarks and outperform the baselines by a significant margin.
BibTeX
@inproceedings{song2020infomax,
  author = {Yuxuan Song and Minkai Xu and Lantao Yu and Hao Zhou and Shuo Shao and Yong Yu},
  title = {{Infomax Neural Joint Source-Channel Coding via Adversarial Bit Flip}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.6041},
}
Figure