Deep-learning-based precoding in multiuser MIMO downlink channels with limited feedback

Kyeongbo Kong, Woo-Jin Song, Moonsik Min

Abstract

We propose a deep learning-based channel estimation, quantization, feedback, and precoding method for downlink multiuser multiple-input and multiple-output (MU-MIMO) systems. In the proposed system, channel estimation and quantization for limited feedback are handled by a receiver deep neural network (DNN). Precoder selection is handled by a transmitter DNN. To emulate the traditional channel quantization, a binarization layer is adopted at each receiver DNN, and the binarization layer is also used to enable end-to-end learning. However, this can lead to inaccurate gradients, which can trap the receiver DNNs at a poor local minimum during training. To address this, we consider knowledge distillation, in which the existing DNNs are jointly trained with an auxiliary transmitter DNN. The use of an auxiliary DNN as a teacher network allows the receiver DNNs to additionally exploit lossless gradients, which is useful in avoiding a poor local minimum. For the same number of feedback bits, our DNN-based precoding scheme can achieve a higher downlink rate compared to conventional linear precoding with codebook-based limited feedback.

Introduction

In MU-MIMO downlink systems using Frequency-Division Duplexing (FDD), the base station (BS) requires accurate Channel State Information (CSI) to perform effective precoding—the process of managing interference to send independent data streams to multiple users simultaneously. In practical scenarios, this CSI must be fed back from users via a limited-rate uplink channel, necessitating quantization into a finite number of bits.

Traditionally, this is achieved using fixed codebooks. While Deep Learning (DL) offers a promising path toward end-to-end optimization of these communication blocks, training such systems is notoriously difficult. To simulate bit-based feedback, the neural network must include a non-differentiable binarization layer. Standard workarounds like Straight-Through Estimators (STE) often introduce gradient noise, which can trap the model in suboptimal local minima during training.

Proposed Deep Learning Framework

The proposed architecture utilizes a dual-DNN setup to jointly optimize channel estimation, feedback quantization, and precoding.

System architecture of the proposed MU-MIMO system including the auxiliary transmitter DNN for knowledge distillation.

Architecture Components

  1. Receiver DNNs: Located at each user's device, these networks process raw pilot signals to estimate the channel and compress the information into a $B$-bit feedback vector using a binarization layer.
  2. Transmitter DNN: Located at the BS, this network aggregates the binary feedback from all users to compute optimal precoding matrices.

Knowledge Distillation (KD) Strategy

The core innovation is the use of Knowledge Distillation (KD) to overcome the "noisy gradient" problem. An Auxiliary Transmitter DNN (the "Teacher") is introduced during training. This teacher network receives the raw, unquantized output from the receiver DNNs. By bypassing the binarization layer, it provides "lossless gradients" that guide the receiver DNNs more effectively. The system follows a joint training procedure where the main network (the "Student") and the auxiliary network are updated iteratively, ensuring the receiver learns to produce features useful for precoding before they are quantized into bits.

Experimental Results

The framework was evaluated using synthetic Rayleigh fading channel data, with 100,000 independent datasets used for both training and testing across various antenna configurations.

Sum rate performance comparison between the proposed KD-aided training and various baselines, including conventional codebook-based methods.

As shown in Figure 3, the proposed KD-aided model consistently achieves higher sum rates than conventional methods like Regularized Block Diagonalization (RBD) and Block Diagonalization (BD) for the same feedback budget. The performance gap is particularly notable as the number of receive antennas increases. Furthermore, the DL-based precoder does not strictly enforce orthogonal columns in the precoding matrix, providing the flexibility needed to find better solutions in high-interference, low-feedback scenarios where perfect orthogonality is unattainable.

Conclusion and Future Work

This work demonstrates that a DL-based approach, bolstered by knowledge distillation, can significantly improve MU-MIMO performance under limited feedback constraints. Future research will focus on enhancing the model's robustness to real-world channel correlations, addressing dynamic user scheduling, and developing architectures that can handle a variable number of users without requiring retraining. Additionally, optimizing these DNNs for real-time inference on 5G/6G hardware remains a key engineering challenge.

Summary updated

Abstract

We propose a deep learning-based channel estimation, quantization, feedback, and precoding method for downlink multiuser multiple-input and multiple-output systems. In the proposed system, channel estimation and quantization for limited feedback are handled by a receiver deep neural network (DNN). Precoder selection is handled by a transmitter DNN. To emulate the traditional channel quantization, a binarization layer is adopted at each receiver DNN, and the binarization layer is also used to enable end-to-end learning. However, this can lead to inaccurate gradients, which can trap the receiver DNNs at a poor local minimum during training. To address this, we consider knowledge distillation, in which the existing DNNs are jointly trained with an auxiliary transmitter DNN. The use of an auxiliary DNN as a teacher network allows the receiver DNNs to additionally exploit lossless gradients, which is useful in avoiding a poor local minimum. For the same number of feedback bits, our DNN-based precoding scheme can achieve a higher downlink rate compared to conventional linear precoding with codebook-based limited feedback.
BibTeX
@article{kong2021deep,
  author = {Kyeongbo Kong and Woo-Jin Song and Moonsik Min},
  title = {{Deep-learning-based precoding in multiuser MIMO downlink channels with limited feedback}},
  journal = {IEEE Trans. Vehicular Technology},
  year = {2021},
  doi = {10.1109/tvt.2021.3110608},
}
Figure