Bit Error and Block Error Rate Training for ML-Assisted Communication

Reinhard Wiesmayr, Gian Marti, Chris Dick, Haochuan Song, Christoph Studer

TL;DR

Most machine learning models for wireless communication are trained using standard classification losses like Binary Cross-Entropy (BCE). This paper reveals that BCE is suboptimal for real-world coded systems because it ignores the relationship between bits in a block and over-prioritizes noisy, low-SNR signals. The authors introduce new "block-aware" loss functions and an "SNR deweighting" technique that significantly improve reliability (BLER) for free.

What Problem Does This Paper Address?

In modern communications, we don't just send individual bits; we send blocks of bits protected by Forward Error Correction (FEC) codes. If even one bit in a block is wrong, the whole block is usually discarded and must be retransmitted. Therefore, the metric that actually matters is the Block Error Rate (BLER), not the Bit Error Rate (BER).

Despite this, most researchers train ML-assisted receivers using BCE loss, which treats every bit as an independent classification task. The authors identify two major gaps:

  1. The BER-BLER Gap: Minimizing bit errors individually (BCE) is mathematically different from minimizing the probability of a block being wrong.
  2. The SNR Imbalance: When training a model to work across a range of Signal-to-Noise Ratios (SNRs), the high-noise (low SNR) samples produce much larger loss values. Consequently, the optimizer spends all its "energy" fixing low-SNR performance while neglecting the high-SNR regime where the system should ideally be most reliable.

Methods

The authors propose two main interventions:

1. BLER-Specific Loss Functions

Instead of summing up the BCE of individual bits, they propose losses that penalize bit errors jointly. Key examples include:

  • Product Loss: A differentiable way to represent the probability that all bits in a block are correct.
  • SmoothMax & LogSumExp: These functions act like a "soft" maximum, focusing the training on the most "uncertain" bit in a block, which is usually the one that causes a block failure.

2. SNR Deweighting

To prevent low-SNR samples from dominating the training, the authors introduce an adaptive weighting scheme. They partition training batches by SNR and track the cumulative loss for each SNR level. In the next epoch, they weight the samples by the inverse of their cumulative loss. This "levels the playing field," ensuring the model cares just as much about a 0.001 error at high SNR as it does about a 0.1 error at low SNR.

Datasets

The paper relies on high-fidelity simulations rather than static datasets, using the NVIDIA Sionna library. They evaluate their methods on two distinct scenarios:

  • 5G MIMO-OFDM: A complex multi-antenna system using a "Deep Unfolded Interleaved Detection and Decoding" (DUIDD) receiver and 5G LDPC codes.
  • SISO AWGN: A simpler single-input single-output channel used to prove that these gains aren't just a fluke of complex architectures.

Key Insights and Ideas

  • BCE is for Marginals: The authors prove that BCE and Mean Squared Error (MSE) learn the posterior marginals (the probability of each bit being 1 given the input). While this is great for BER, it ignores the conditional dependencies between bits that are crucial for decoding a full block.
  • The "Aha" Moment on SNR: Naive training on an SNR range is essentially "training for the worst-case scenario." While this sounds robust, it actually leads to a "floor" in performance where the model never learns to be ultra-precise when the signal is clean.
  • Free Gains: These improvements don't require more complex neural networks or more training data; they are simply a result of changing the objective function.

Key Results

  • BLER Improvement: In the 5G MIMO-OFDM experiment, using BLER-specific loss functions improved performance by 0.62 dB at a 1% BLER target compared to standard BCE.
  • Robustness via Deweighting: SNR deweighting outperformed naive training by 0.54 dB. More importantly, it prevented the "complete breakdown" at high SNRs that often occurs when a model is only trained on noisy data.
  • Consistency: The results held true across different receiver architectures, proving that the choice of loss function is a fundamental design decision in ML-comms.

Limitations and Future Work

  • Block Length Constraints: The authors note that while their losses work for standard FEC blocks, the "Categorical Cross-Entropy" (an alternative joint loss) only works for very short blocks (around 8 bits) due to exponential complexity. Their proposed losses (like Product or Max) scale better but may still face gradient challenges in extremely large blocks.
  • Vanishing Gradients: The Product Loss, while theoretically sound, can suffer from vanishing gradients because it is bounded between 0 and 1. The authors suggest the Max-based smooth approximations as a more practical alternative.
  • Hardware Implementation: The paper focuses on training logic; future work could explore how these different loss functions affect the actual hardware complexity of the resulting "unfolded" receivers.

Summary updated

BibTeX
@inproceedings{wiesmayr2023bit,
  author = {Reinhard Wiesmayr and Gian Marti and Chris Dick and Haochuan Song and Christoph Studer},
  title = {{Bit Error and Block Error Rate Training for ML-Assisted Communication}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095841},
}
Figure