CRC-Aided Learned Ensembles of Belief-Propagation Polar Decoders

Tomer Raviv, Alon Goldmann, Ofek Vayner, Yair Be'ery, Nir ShlezingerICASSP 2024 · DOI

TL;DR

This paper introduces a novel polar decoder that combines the strengths of traditional belief propagation (BP) decoding with modern deep learning techniques. By creating an ensemble of specialized neural decoders and using CRC codes for efficient selection, they achieve significant improvements in error correction performance with manageable complexity and latency, making it suitable for real-time applications.

What Problem Does This Paper Address?

Modern wireless communication systems, like those in 5G, demand high reliability, low latency, and efficient resource usage. Error-correction codes (ECCs) are crucial for reliability, with polar codes being a prominent choice due to their theoretical capacity-approaching properties. However, decoding polar codes, especially for large block sizes, is computationally challenging. Existing methods like Successive Cancellation List (SCL) offer good performance but suffer from high latency. Belief Propagation (BP) decoders are faster and more parallelizable but generally less accurate than SCL. While deep learning has been applied to improve BP decoders, these methods often increase complexity and latency, negating BP's advantages. This paper aims to bridge this gap by developing a decoder that leverages the speed of BP, the accuracy of deep learning ensembles, and the error-checking capabilities of CRC codes, all while maintaining low latency and complexity suitable for practical systems.

Methods

The core of this work is a CRC-Aided Learned Ensemble of Belief-Propagation (BP) Decoders. The approach builds upon Weighted Belief Propagation (WBP), a variant of BP where message passing is weighted to improve convergence.

Here's a breakdown of the key techniques:

  • Ensemble of WBP Decoders: Instead of a single decoder, the system employs multiple WBP decoders, each trained to specialize in decoding different types of errors or data patterns. This "divide and conquer" strategy aims to leverage the diversity of individual decoders for better overall performance.
  • Gated Ensemble Architecture: To maintain low latency, a standard BP decoder (denoted $F_0$) acts as a "gatekeeper." If this initial decoder successfully decodes the message (verified by a CRC check), its output is used. Only if the CRC check fails, indicating a potential error, is the message passed to the more computationally intensive ensemble of learned WBP decoders ($F_1, \dots, F_a$).
  • CRC-Aided Selection: Cyclic Redundancy Check (CRC) codes, a standard component in 5G, are used in novel ways. They serve not only to verify the correctness of a decoded word but also as a mechanism to diversify the training data for the ensemble members. By partitioning the training data based on the CRC remainder's most significant bits, each ensemble member is trained on a distinct subset of error patterns. This CRC-based partitioning is a key innovation for creating diverse and effective ensemble members.
  • Model-Based Deep Learning: The WBP decoders are "learned" using deep learning. This involves training the neural weights ($\gamma$) of the WBP algorithm to minimize a Binary Cross Entropy (BCE) loss function, effectively teaching the decoder to map received noisy signals to the correct transmitted bits.

Datasets

The paper describes training and evaluation using simulated data.

  • Training Data: The training data is generated by simulating the transmission of labeled message words through an Additive White Gaussian Noise (AWGN) channel. The dataset is then partitioned into subsets ($D^{(i)}$) based on the MSBs of the CRC remainder of the received word, as described in equation (5). This partitioning is crucial for training diverse ensemble members.
  • Validation Data: For evaluation, a validation dataset is used, composed of SNR values from 1dB to 4dB, with 0.5dB spacing. At least 500 errors are accumulated at each point to ensure statistically significant results.

Key Insights and Ideas

  • CRC as a Diversification Tool: The most significant insight is the innovative use of CRC codes not just for error detection but as a principled method to partition training data and create diverse, specialized decoders for an ensemble. This moves beyond simple random permutations or synthetic noise for diversity.
  • Gated Ensemble for Latency: The gated architecture is a clever way to balance performance gains with latency. The fast initial BP decoder handles most cases, only invoking the more complex ensemble when necessary, thus minimizing the overall latency overhead.
  • Hybrid Approach Synergy: The paper demonstrates that combining traditional coding theory (CRC, BP) with modern machine learning (deep learning ensembles) can yield performance superior to either approach alone, especially for challenging tasks like polar code decoding.
  • Practicality for Long Codes: The proposed method shows promise for long polar codes, where many purely neural or complex list-decoding approaches struggle due to computational constraints.

Key Results

  • Performance Gains: The proposed CRC-aided learned ensemble decoder achieves gains of approximately 0.25dB in Frame Error Rate (FER) across various code lengths (128, 256, and 512) in medium-to-high Signal-to-Noise Ratios (SNRs). Gains of around 0.1dB are observed in the low SNR regime.
  • Scalability: Increasing the number of ensemble members (from $\alpha=2$ to $\alpha=8$) leads to further, albeit diminishing, FER improvements (around 0.05dB for longer codes).
  • Manageable Complexity and Latency: The analysis shows that the average latency, due to the gated approach and parallel ensemble processing, only induces a minor increase compared to a single BP decoder, especially at higher SNRs where the gating decoder is more likely to succeed. The number of operations (FLOPS) is shown to be significantly lower than a non-gated ensemble or a high-iteration WBP decoder, particularly at higher SNRs.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that gains in the low SNR regime are harder to achieve, which is a common challenge for neural decoding approaches.
  • The current ensemble relies on partitioning data based on CRC remainder MSBs, which assumes $\log_2 \alpha$ is an integer.
  • Future Work:
    • The authors suggest further improvements by increasing data diversity (e.g., incorporating codeword Hamming distance) or architectural diversity (e.g., using full neural network decoders or Tanner-based decoders as ensemble members).
  • Exploring extensions to other types of ECCs or channel models is also a natural next step.

Summary updated

BibTeX
@inproceedings{raviv2024crc,
  author = {Tomer Raviv and Alon Goldmann and Ofek Vayner and Yair Be'ery and Nir Shlezinger},
  title = {{CRC-Aided Learned Ensembles of Belief-Propagation Polar Decoders}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10447608},
}
Figure