Deep Learning Based Near-Orthogonal Superposition Code for Short Message Transmission

Chenghong Bian, Mingyu Yang, Chin-Wei Hsu, Hun-Seok Kim

Abstract

Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning based near-orthogonal superposition (NOS) coding scheme is proposed for reliable transmission of short messages in the additive white Gaussian noise (AWGN) channel for mMTC applications. Similar to recent hyper-dimensional modulation (HDM), the NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector for transmission. The NOS decoder first estimates the information vectors and then performs a cyclic redundancy check (CRC)-assisted K-best tree-search algorithm to further reduce the packet error rate. The proposed NOS encoder and decoder are deep neural networks (DNNs) jointly trained as an auto encoder and decoder pair to learn a new NOS coding scheme with near-orthogonal codewords. Simulation results show the proposed deep learning-based NOS scheme outperforms HDM and Polar code with CRC-aided list decoding for short(32-bit) message transmission.

Introduction

As we move toward "Massive Machine Type Communication" (mMTC)—the backbone of smart cities, health monitoring, and industrial IoT—we face a unique engineering challenge: short messages. Traditional error-correction codes, such as LDPC or Turbo codes, are designed for long data streams. When message sizes shrink to just a few dozen bits, these codes become inefficient and suffer from high error floors. While Polar codes are currently the gold standard for short packets in 5G, they still do not reach the theoretical limits of channel capacity. This paper proposes a Near-Orthogonal Superposition (NOS) framework using Deep Learning to design a joint coding and modulation scheme that is more robust and easier to decode than existing methods like Hyper-Dimensional Modulation (HDM).

Proposed Method: The NOS Framework

The NOS framework treats the encoder and decoder as a single "auto-encoder" unit trained end-to-end.

  1. The Encoder (Transmitter): The input message is split into $V$ small segments. Each segment is converted into a "one-hot" vector and fed into a dedicated Deep Neural Network (DNN) encoder. These encoders produce high-dimensional vectors that are superimposed into a single signal for transmission over an AWGN channel.
  2. Emergent Near-Orthogonality: During training, the DNNs naturally learn to produce codewords that are nearly orthogonal to one another, minimizing interference when superimposed. This near-orthogonality (roughly -30dB) emerges naturally from the goal of minimizing transmission errors.
  3. The Decoder (Receiver): While initially using a DNN decoder, the authors found that a mathematically derived Maximum A-Posteriori (MAP) decoder performed just as well with lower complexity.
  4. CRC-Assisted K-Best Decoding: To further enhance performance, a Cyclic Redundancy Check (CRC) and a "K-best" tree-search algorithm are employed. The decoder maintains a list of the $K$ most likely candidates and uses CRC bits to verify the correct message.
Architecture of the proposed Near-Orthogonal Superposition (NOS) encoder and decoder.

Performance Evaluation

The models were trained on $5 \times 10^5$ samples per epoch using an ADAM optimizer and evaluated against 3GPP Polar codes and HDM. A key finding is the trade-off between the number of encoders ($V$) and input dimensions ($M$); fewer encoders with larger input dimensions generally lead to better error performance, though very large $M$ values increase training difficulty.

For a 32-bit message, the NOS scheme achieved a 0.5 dB gain over Polar codes, representing a significant improvement in power efficiency. Furthermore, the MAP-based decoding process demonstrates lower computational complexity and shorter run-times on standard CPUs compared to HDM or Polar list decoding.

Bit Error Rate (BER) performance comparison between Neural Network (NN) and Maximum A-Posteriori (MAP) decoders for different configurations.

Conclusion

The proposed NOS scheme offers a promising alternative for reliable short message transmission in mMTC. While the current approach faces scaling issues for longer messages due to the exponential growth of input size ($M = 2^m$), it demonstrates superior reliability and lower complexity for short packets. Future work will investigate more efficient DNN architectures, such as Transformers, and more complex training scenarios involving real-world wireless fading environments.

Summary updated

Abstract

Massive machine type communication (mMTC) has attracted new coding schemes optimized for reliable short message transmission. In this paper, a novel deep learning based near-orthogonal superposition (NOS) coding scheme is proposed for reliable transmission of short messages in the additive white Gaussian noise (AWGN) channel for mMTC applications. Similar to recent hyper-dimensional modulation (HDM), the NOS encoder spreads the information bits to multiple near-orthogonal high dimensional vectors to be combined (superimposed) into a single vector for transmission. The NOS decoder first estimates the information vectors and then performs a cyclic redundancy check (CRC)-assisted K-best tree-search algorithm to further reduce the packet error rate. The proposed NOS encoder and decoder are deep neural networks (DNNs) jointly trained as an auto encoder and decoder pair to learn a new NOS coding scheme with near-orthogonal codewords. Simulation results show the proposed deep learning-based NOS scheme outperforms HDM and Polar code with CRC-aided list decoding for short(32-bit) message transmission.
BibTeX
@inproceedings{bian2022deep,
  author = {Chenghong Bian and Mingyu Yang and Chin-Wei Hsu and Hun-Seok Kim},
  title = {{Deep Learning Based Near-Orthogonal Superposition Code for Short Message Transmission}},
  booktitle = {ICC},
  year = {2022},
  url = {https://openreview.net/forum?id=AHB9-pfuMH},
}
Figure