Abstract
Most of today's communication systems are designed to target reliable message recovery after receiving the entire encoded message (codeword). However, in many practical scenarios, the transmission process may be interrupted before receiving the complete codeword. This paper proposes a novel rateless autoencoder (AE)-based code design suitable for decoding the transmitted message before the noisy codeword is fully received. Using particular dropout strategies applied during the training process, rateless AE codes allow to trade off between decoding delay and reliability, providing a graceful improvement of the latter with each additionally received codeword symbol. The proposed rateless AEs significantly outperform the conventional AE designs for scenarios where it is desirable to trade off reliability for lower decoding delay.
Introduction
In classical communication theory, receivers are typically expected to process a complete codeword before attempting to decode the original message. However, real-world "dying channels"—where transmissions are abruptly interrupted—present a significant challenge. Such interruptions occur frequently in satellite communications, energy-harvesting sensor networks, or cognitive radio environments where primary users may reclaim spectrum unexpectedly.
Standard fixed-rate codes, including conventional neural autoencoders (C-AEs), are optimized for a specific block length. These systems often exhibit a "cliff effect": if the channel terminates early and the receiver obtains only a partial codeword (e.g., 70% of symbols), the block error rate (BLER) typically skyrockets toward 1.0. While classical rateless codes like Spinal or Fountain codes address this, they can be difficult to optimize for short block lengths or specific hardware. This research introduces Rateless Autoencoder (R-AE) codes, leveraging deep learning to provide flexible, end-to-end optimized error correction for unpredictable channel conditions.
Proposed Methodology
The core of the proposed system is a deep autoencoder architecture where the encoder maps a message $m$ to a sequence of symbols $x$, and the decoder maps noisy received symbols $y$ back to an estimate $\hat{m}$. The innovation lies in the integration of Channel Dropout Strategies during the training phase to bake rate-compatibility directly into the network weights.
Two primary dropout models were investigated:
- Tail Erasures: A dropout mask zeros out the end of a codeword, simulating a channel that "dies" at a random point. This forces the encoder to prioritize information density in the earlier symbols.
- Random Erasures: Symbols are dropped randomly across the codeword to simulate intermittent deep fades or packet loss.
The system is trained end-to-end using the Adam optimizer and a cross-entropy loss function. By randomizing the dropout mask batch-by-batch, the network learns a representation that allows for message recovery with varying levels of certainty depending on the number of symbols received.
Experimental Evaluation
The performance was evaluated using a synthetic dataset standard for physical layer research:
- Message Space: $M = 4096$ unique messages ($k=12$ bits).
- Codeword Length: A maximum length of $n = 24$ symbols.
- Channel Model: Additive White Gaussian Noise (AWGN) combined with erasure models.
- Statistical Rigor: $10^5$ messages were used for training and $10^6$ for testing to ensure accurate BLER measurements.
Results and Discussion
The R-AE demonstrates a significant advantage in early decoding scenarios. While a conventional AE is highly specialized for its full block length, it fails almost entirely when symbols are missing. In contrast, the R-AE exhibits "graceful degradation," where reliability improves incrementally with every additional symbol received.
Key findings include:
- Superior Early Decoding: At a received length of $r=15$ symbols (out of 24), the R-AE maintains a usable error rate while the C-AE fails completely.
- The Trade-off: The C-AE performs slightly better only when the entire codeword is received, as it is specialized for that exact length. The R-AE is vastly superior at all intermediate lengths.
- Power Constraints: The authors noted that an average power constraint during training is vital. A fixed per-symbol power constraint can inhibit the rateless property by over-specializing the decoder to specific energy levels.
Conclusion
Rateless Autoencoders provide a robust solution for communication over unpredictable channels by treating dropout as a structural tool rather than just a regularization technique. While current research focuses on short block lengths ($n=24$), the results prove that neural codes can effectively trade off decoding delay for reliability. Future work will explore scaling these architectures to longer sequences and moving toward truly infinite "fountain-style" neural codes.
Summary updated