Score Based Error Correcting Code Decoder

Alon Helvits, Eliya Nachmani

Introduction

Error-correcting codes (ECC) are the backbone of reliable digital communication, designed to recover data corrupted by channel noise. While optimal decoding is mathematically NP-hard, practical systems typically rely on iterative algorithms like Belief Propagation. Recently, deep learning has emerged as a promising "model-free" alternative. However, existing neural decoders face two major hurdles: they often discard the "sign" of the received signal (losing vital geometric information) and require precise Signal-to-Noise Ratio (SNR) estimation at inference time, which is notoriously difficult in fluctuating real-world channels.

To address these challenges, researchers introduce the Score-Based Error-Correcting Code Decoder (SB-ECC), a novel neural decoder that treats error correction as a continuous-time denoising process. By solving a deterministic Ordinary Differential Equation (ODE) directly on raw, signed channel observations, SB-ECC achieves state-of-the-art performance across various code families without requiring prior knowledge of the channel's SNR.

Methodology: Decoding as Continuous-Time Denoising

SB-ECC leverages score-based generative modeling—a relative of diffusion models—to frame the noisy channel as a forward diffusion process that corrupts a clean codeword. Decoding becomes the reverse process: iteratively removing noise to recover the valid codeword.

  • Probability-Flow ODE (PF-ODE): Instead of random sampling, SB-ECC uses a deterministic ODE to "flow" the noisy observation back toward a valid codeword, guided by parity constraints.
  • Signed Input: Unlike previous state-of-the-art models (e.g., DDECC or ECCT) that use magnitude-only preprocessing, SB-ECC operates directly on the raw, signed signal. This preserves the geometric structure of the codeword space.
  • Time-Unconditional Training: The model is trained across a wide range of noise levels without being conditioned on the specific SNR. It learns to infer the "effective noise" directly from the received signal, bypassing the need for a separate SNR estimation block.
  • Architecture & Solvers: SB-ECC utilizes the CrossMPT (Cross-attention Message-Passing Transformer) architecture to mimic the code's parity-check graph. Furthermore, it implements advanced ODE solvers, such as the DPM-Solver, to reduce the number of integration steps and lower latency.

Experimental Setup

The model was evaluated on a comprehensive suite of standard linear block codes, including BCH, Polar, LDPC, MacKay, and CCSDS codes, with lengths ranging from 31 to 529 bits. It was also tested in a Rayleigh Fading Channel to assess robustness in more complex, non-Gaussian noise environments.

Key Insights and Results

  • State-of-the-Art Accuracy: Across 42 code/SNR settings, SB-ECC achieved the best Bit Error Rate (BER) in 39 scenarios. It delivered an average SNR gain of 0.17 dB and a maximum gain of 0.46 dB over the strongest existing neural baselines.
  • The Importance of Sign: Ablation studies revealed that preserving the sign of the input is critical for score-based models. Models trained on magnitude-only data failed to surpass simple hard-decision baselines.
  • Efficiency and Early Exiting: By checking parity constraints (the syndrome) after each ODE step, the decoder can halt early once a valid codeword is found, drastically reducing computational cost at high SNRs.
  • Solver Flexibility: The ODE framework allows users to swap solvers at test time for a direct latency-accuracy trade-off. Swapping the standard Euler solver for the DPM-Solver reduced end-to-end decoding time by an average of 8.86% (and up to 12.82%) while preserving the exact same error-rate performance.
  • Zero-Shot Robustness: Trained on standard AWGN noise, SB-ECC maintained top performance when tested on a Rayleigh fading channel, demonstrating strong generalization to varying channel conditions.

Limitations and Future Directions

While SB-ECC is faster than previous iterative diffusion decoders, it remains slower than "one-step" neural decoders due to the multiple network passes required by the ODE solver. Furthermore, scaling Transformer-based attention mechanisms to block lengths of thousands of bits remains computationally challenging.

Future work will explore Consistency Models to distill the multi-step ODE trajectory into a single-step decoder. This approach aims to combine the high accuracy and robustness of SB-ECC with the ultra-low latency of one-step models, paving the way for practical deployment in next-generation communication systems.

Summary updated

Abstract

Error-correcting codes enable reliable communication, yet practical soft decoding remains challenging across code families and block lengths. We propose SB-ECC, a score-based decoder that casts decoding as continuous-time denoising. A neural denoiser defines a probability-flow ordinary differential equation (ODE) that iteratively updates the noisy channel observation toward a valid codeword, guided by parity constraints. The model is trained across noise levels without time/SNR conditioning, enabling inference without SNR estimation and supporting a direct latency--accuracy trade-off controlled by the ODE solver budget. We use the raw signed channel observation as input for learning a continuous denoising field. Across $42$ code/SNR settings, SB-ECC achieves the best BER in $39/42$ entries, with an average SNR gain of $0.17$\,dB and a maximum gain of $0.46$\,dB over the strongest competing baseline, we showed that swapping the solver from Euler to DPM preserves $-\ln(\mathrm{BER})$ while reducing end-to-end decoding time by $8.86\%$ on average (up to $12.82\%$).
BibTeX
@inproceedings{helvits2026score,
  author = {Alon Helvits and Eliya Nachmani},
  title = {{Score Based Error Correcting Code Decoder}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=uQiLoAF0bM},
}
Figure