Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels

Yihan Jiang, Hyeji Kim, Himanshu Asnani, Sreeram Kannan, Sewoong Oh, Pramod Viswanath

Abstract

Designing codes that combat noise in a communication medium has remained a significant area of research in information theory and wireless communications. While asymptotically optimal channel codes exist for canonical models like Additive White Gaussian Noise (AWGN), many real-world, non-canonical channel settings lack optimal codes. In these cases, classical codes are often adapted via sub-optimal heuristics. This work introduces Turbo Autoencoder (TurboAE), a fully end-to-end jointly trained neural encoder and decoder. TurboAE approaches state-of-the-art performance under canonical channels at moderate block lengths and outperforms existing codes in non-canonical settings. This demonstrates that channel coding design can be automated via deep learning with near-optimal performance.

Introduction

In digital communications, channel coding adds redundancy to messages to ensure recovery after transmission through a noisy medium. For over six decades, mathematicians have refined codes like Turbo, Polar, and LDPC for the canonical AWGN channel. However, real-world channels are often non-stationary, bursty, or heavy-tailed. In these environments, human-designed heuristics often fail to achieve optimality.

Previous attempts to apply deep learning to channel coding struggled to scale. Standard neural architectures, such as simple CNNs or RNNs, possess "local memory" that prevents them from learning the long-range dependencies necessary for high reliability over moderate to long block lengths. TurboAE addresses this by integrating classical communication principles—specifically interleaving—into a deep learning framework.

Architecture and Methodology

TurboAE mimics the "Turbo principle" used in 3G/4G standards by employing an Interleaved CNN Architecture.

The Encoder and Decoder

The encoder consists of multiple CNN blocks separated by interleavers (pseudo-random permutations). This structure forces the network to spread information across the entire data block, effectively overcoming the local memory limitations of standard CNNs. The decoder mirrors this structure, utilizing multiple iterations of CNN-based decoding and de-interleaving to successively refine the message estimate.

TurboAE Encoder Architecture featuring interleaving blocks to capture long-range dependencies.

Training Innovations

To achieve high reliability, the authors implemented several training strategies:

  • TurboAE-binary: Uses a hard power constraint (bits are -1 or +1), trained via a Straight-Through Estimator (STE) to handle non-differentiable functions.
  • Large Batch Training: Batches exceeding 500 examples are used to average out stochastic channel noise.
  • Alternating Optimization: The encoder and decoder are trained in alternating steps to stabilize the learning process.

Experiments and Results

The performance of TurboAE was evaluated across several synthetic channel models, including AWGN, Additive T-distribution Noise (ATN), Markovian-AWGN (bursty noise), and Rayleigh Fading.

Performance Benchmarks

In the canonical AWGN channel with a block length of $K=100$, TurboAE approaches the performance of state-of-the-art LDPC and Polar codes. More importantly, in non-canonical settings like the heavy-tailed ATN channel, TurboAE-continuous significantly outperforms classical Turbo codes and neural decoders designed for fixed classical encoders.

Bit Error Rate (BER) performance comparison between TurboAE and state-of-the-art classical codes.

In Rayleigh fading environments, common in mobile wireless, TurboAE-binary demonstrated superior reliability compared to LDPC, TBCC, and standard Turbo codes across various Signal-to-Noise Ratio (SNR) levels.

Discussion and Future Work

TurboAE proves that interleaving is a critical structural prior for neural channel codes. By allowing the network to "discover" coding strategies through data-driven training, TurboAE offers a flexible alternative to hard-coded classical algorithms.

Despite these successes, challenges remain in scaling to very long block lengths (thousands of bits) due to computational and memory constraints. Additionally, training at high SNR remains difficult because error events become too rare for standard gradient descent to capture effectively. Future research will focus on optimizing these architectures for real-time hardware implementation through pruning and quantization.

Summary updated

Abstract

Designing codes that combat the noise in a communication medium has remained a significant area of research in information theory as well as wireless communications. Asymptotically optimal channel codes have been developed by mathematicians for communicating under canonical models after over 60 years of research. On the other hand, in many non-canonical channel settings, optimal codes do not exist and the codes designed for canonical models are adapted via heuristics to these channels and are thus not guaranteed to be optimal. In this work, we make significant progress on this problem by designing a fully end-to-end jointly trained neural encoder and decoder, namely, Turbo Autoencoder (TurboAE), with the following contributions: ($a$) under moderate block lengths, TurboAE approaches state-of-the-art performance under canonical channels; ($b$) moreover, TurboAE outperforms the state-of-the-art codes under non-canonical settings in terms of reliability. TurboAE shows that the development of channel coding design can be automated via deep learning, with near-optimal performance.
BibTeX
@inproceedings{jiang2019turbo,
  author = {Yihan Jiang and Hyeji Kim and Himanshu Asnani and Sreeram Kannan and Sewoong Oh and Pramod Viswanath},
  title = {{Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=a0BPk7a9wMe},
}
Figure