Abstract
Designing codes that combat noise in a communication medium has remained a significant area of research in information theory and wireless communications. While asymptotically optimal channel codes exist for canonical models like Additive White Gaussian Noise (AWGN), many real-world, non-canonical channel settings lack optimal codes. In these cases, classical codes are often adapted via sub-optimal heuristics. This work introduces Turbo Autoencoder (TurboAE), a fully end-to-end jointly trained neural encoder and decoder. TurboAE approaches state-of-the-art performance under canonical channels at moderate block lengths and outperforms existing codes in non-canonical settings. This demonstrates that channel coding design can be automated via deep learning with near-optimal performance.
Introduction
In digital communications, channel coding adds redundancy to messages to ensure recovery after transmission through a noisy medium. For over six decades, mathematicians have refined codes like Turbo, Polar, and LDPC for the canonical AWGN channel. However, real-world channels are often non-stationary, bursty, or heavy-tailed. In these environments, human-designed heuristics often fail to achieve optimality.
Previous attempts to apply deep learning to channel coding struggled to scale. Standard neural architectures, such as simple CNNs or RNNs, possess "local memory" that prevents them from learning the long-range dependencies necessary for high reliability over moderate to long block lengths. TurboAE addresses this by integrating classical communication principles—specifically interleaving—into a deep learning framework.
Architecture and Methodology
TurboAE mimics the "Turbo principle" used in 3G/4G standards by employing an Interleaved CNN Architecture.
The Encoder and Decoder
The encoder consists of multiple CNN blocks separated by interleavers (pseudo-random permutations). This structure forces the network to spread information across the entire data block, effectively overcoming the local memory limitations of standard CNNs. The decoder mirrors this structure, utilizing multiple iterations of CNN-based decoding and de-interleaving to successively refine the message estimate.
Training Innovations
To achieve high reliability, the authors implemented several training strategies:
- TurboAE-binary: Uses a hard power constraint (bits are -1 or +1), trained via a Straight-Through Estimator (STE) to handle non-differentiable functions.
- Large Batch Training: Batches exceeding 500 examples are used to average out stochastic channel noise.
- Alternating Optimization: The encoder and decoder are trained in alternating steps to stabilize the learning process.
Experiments and Results
The performance of TurboAE was evaluated across several synthetic channel models, including AWGN, Additive T-distribution Noise (ATN), Markovian-AWGN (bursty noise), and Rayleigh Fading.
Performance Benchmarks
In the canonical AWGN channel with a block length of $K=100$, TurboAE approaches the performance of state-of-the-art LDPC and Polar codes. More importantly, in non-canonical settings like the heavy-tailed ATN channel, TurboAE-continuous significantly outperforms classical Turbo codes and neural decoders designed for fixed classical encoders.
In Rayleigh fading environments, common in mobile wireless, TurboAE-binary demonstrated superior reliability compared to LDPC, TBCC, and standard Turbo codes across various Signal-to-Noise Ratio (SNR) levels.
Discussion and Future Work
TurboAE proves that interleaving is a critical structural prior for neural channel codes. By allowing the network to "discover" coding strategies through data-driven training, TurboAE offers a flexible alternative to hard-coded classical algorithms.
Despite these successes, challenges remain in scaling to very long block lengths (thousands of bits) due to computational and memory constraints. Additionally, training at high SNR remains difficult because error events become too rare for standard gradient descent to capture effectively. Future research will focus on optimizing these architectures for real-time hardware implementation through pruning and quantization.
Summary updated