Abstract
Traditional communication systems rely on the separation of source and channel coding, which is often sub-optimal for finite-length data blocks in real-world applications. This paper presents Infomax Adversarial-Bit Flip (IABF), a joint source-channel coding (JSCC) framework that leverages mutual information maximization and adversarial regularization. By maximizing the information content of binary codewords and training against worst-case bit-flip scenarios, IABF achieves superior robustness and reconstruction accuracy compared to existing neural JSCC methods.
Introduction
In classical information theory, Shannon’s separation theorem posits that source coding (compression) and channel coding (error correction) can be treated independently without loss of optimality. However, this optimality holds only in the limit of infinite block lengths. For modern low-latency applications like IoT or real-time streaming, finite-length constraints make joint source-channel coding (JSCC) more effective.
Recent neural JSCC approaches, such as NECST, utilize autoencoders to learn end-to-end mappings. Despite their promise, these models often lack robustness to channel noise and fail to produce highly informative codewords. The IABF framework addresses these limitations by explicitly optimizing for informativeness and worst-case error resilience.
Proposed Method: IABF
The IABF architecture introduces two primary innovations to the standard autoencoder-based JSCC pipeline:
- Infomax Encoder: To ensure the binary codeword $y$ captures maximal information about the input $x$, the model maximizes the Mutual Information (MI) between them. Since direct MI computation is intractable for high-dimensional bit-strings, IABF employs a Total Correlation (TC) penalty. A discriminator network is trained to distinguish between the joint distribution of bits and the product of their marginals, encouraging the encoder to produce informative, independent bits.
- Adversarial Bit Flip Regularization: To enhance decoder robustness, IABF moves beyond random noise injection. It identifies "vulnerable" bits by calculating the gradient of the reconstruction loss with respect to the codeword. By "virtually" flipping these critical bits during training, the decoder learns to reconstruct data even under adversarial noise conditions.
Experimental Evaluation
The performance of IABF was evaluated on several image benchmarks, including BinaryMNIST, MNIST, Omniglot, and CIFAR10. These datasets represent varying levels of complexity for testing reconstruction quality across different noise regimes.
Key Results
Experimental results demonstrate that IABF consistently outperforms state-of-the-art baselines like NECST:
- Reconstruction Accuracy: IABF achieved lower $L^2$ reconstruction errors across all tested noise levels (ranging from 10% to 40% bit-flip probability).
- Training Stability: Unlike previous models that exhibit "spiky" training curves and susceptibility to overfitting, IABF maintains a stable learning trajectory throughout the training process.
- Information Distribution: The adversarial mechanism forces the encoder to distribute information uniformly across the codeword, preventing the model from over-relying on a few "fragile" bits that would cause catastrophic failure if flipped.
Conclusion
IABF represents a significant step forward in neural joint source-channel coding. By integrating information-theoretic objectives with adversarial training, it provides a robust solution for data transmission over noisy channels. Future work includes extending the adversarial bit-flip concept to non-binary channels and exploring active learning strategies to further optimize the training process against complex noise patterns.
Summary updated