Abstract
Semantic communication (SemCom) shifts the focus from transmitting every bit perfectly to preserving the task-relevant meaning of information. While Deep Joint Source-Channel Coding (Deep-JSCC) has shown promise, it often relies on continuous real-valued outputs that are incompatible with discrete digital systems. This paper introduces E2EC, a novel end-to-end coding framework that bridges this gap. By decomposing the encoding process into separate length and content modules and utilizing policy gradients to optimize through non-differentiable operations, E2EC achieves efficient, variable-length bit-level rate control. Experimental results on the MNIST dataset demonstrate that E2EC significantly outperforms traditional fixed-length digital JSCC methods and approaches theoretical rate-distortion bounds.
Introduction
Traditional communication systems are designed to minimize bit error rates, treating all bits as equally important. Semantic Communication (SemCom) challenges this by prioritizing the transmission of the meaning or task-relevant information. Deep-JSCC has emerged as a powerful tool for SemCom, yet it faces a significant hurdle: digital incompatibility.
Most Deep-JSCC models output continuous vectors, whereas real-world digital systems require discrete binary codewords. Furthermore, while digital systems benefit from variable-length coding—allocating more bits to complex data and fewer to simple data—training neural networks for this is difficult. Operations such as "sampling a bit" or "truncating a sequence" are non-differentiable, preventing standard backpropagation. This paper addresses these challenges by proposing a truly end-to-end trainable system for variable-length binary semantic transmission.
The E2EC Framework
System Architecture
The E2EC framework relies on a structural decomposition of the encoding process. Rather than using a single monolithic network, the encoder is split into two functional modules:
- Length Module ($f_l$): Determines the optimal number of bits ($l$) to transmit based on the input data's complexity.
- Content Module ($f_{\tilde{z}}$): Generates a maximum-length binary sequence.
The final transmitted codeword is the content sequence truncated to the length specified by the length module. This design inherently supports variable-rate transmission and simplifies the output space from exponential complexity ($2^{R_{max}}$) to linear complexity relative to the maximum code length.
Optimization via Policy Gradients
To handle the non-differentiable nature of discrete sampling and truncation, the authors employ Policy Gradient methods (score function estimators) from Reinforcement Learning. This allows the model to estimate gradients and update parameters even when the channel noise is unknown or the operations are discrete. Additionally, the framework extends Information Bottleneck (IB) theory to explicitly account for noisy channel conditions, optimizing the tradeoff between compression (rate) and semantic accuracy (distortion).
Experimental Evaluation
The framework was evaluated using the MNIST dataset for a semantic classification task (identifying digits 0-9) over a Binary Symmetric Channel (BSC).
Key Results and Insights
Experimental data reveals several critical advantages of the E2EC approach:
- Rate-Distortion Efficiency: E2EC's coding efficiency significantly exceeds traditional digital Deep-JSCC baselines. As the bit rate increases, the performance closely approaches the theoretical Rate-Distortion Bound (RDBO).
- Progressive Refinement: A key discovery is that the model naturally learns to pack the most "informative" semantic bits at the beginning of the codeword. As shown in the bit-block importance analysis, discarding later bits causes significantly less semantic distortion than discarding early bits.
- Robustness and Convergence: The system maintains high inference accuracy (above 98%) even under high channel error probabilities by intelligently adapting code length. Furthermore, the $R-D$ pairs converge predictably on the information plane, validating the stability of the IB-based training process.
Conclusion
E2EC provides a robust path forward for integrating semantic communication into practical digital infrastructures. By solving the differentiability problem of variable-length binary coding, it enables dynamic rate control that is both efficient and robust to channel noise. While currently demonstrated on MNIST, the framework's linear complexity and progressive refinement properties suggest it is well-suited for scaling to more complex, high-dimensional data in future semantic communication systems.
Summary updated