Knowledge Distillation-aided End-to-End Learning for Linear Precoding in Multiuser MIMO Downlink Systems with Finite-Rate Feedback

Kyeongbo Kong, Woo‐Jin Song, Moonsik Min

Abstract

We propose a deep learning-based channel estimation, quantization, feedback, and precoding method for downlink multiuser multiple-input and multiple-output systems. In the proposed system, channel estimation and quantization for limited feedback are handled by a receiver deep neural network (DNN). Precoder selection is handled by a transmitter DNN. To emulate the traditional channel quantization, a binarization layer is adopted at each receiver DNN, and the binarization layer is also used to enable end-to-end learning. However, this can lead to inaccurate gradients, which can trap the receiver DNNs at a poor local minimum during training. To address this, we consider knowledge distillation, in which the existing DNNs are jointly trained with an auxiliary transmitter DNN. The use of an auxiliary DNN as a teacher network allows the receiver DNNs to additionally exploit lossless gradients, which is useful in avoiding a poor local minimum. For the same number of feedback bits, our DNN-based precoding scheme can achieve a higher downlink rate compared to conventional linear precoding with codebook-based limited feedback.

Introduction

In Multiuser Multiple-Input Multiple-Output (MU-MIMO) systems, the base station (BS) requires accurate Channel State Information (CSI) to perform effective precoding and minimize inter-user interference. In Frequency-Division Duplexing (FDD) systems, users must estimate their channels and provide feedback to the BS. Due to bandwidth constraints, this feedback is typically limited to a finite number of bits.

Traditional methods rely on codebook-based quantization. While Deep Learning (DL) offers a promising alternative for end-to-end optimization, the necessary binarization of feedback data introduces non-differentiable operations (e.g., the sign function). This leads to the "vanishing gradient" problem, where zero gradients during backpropagation prevent effective training and trap models in suboptimal local minima.

Proposed Methodology

The authors introduce a joint training architecture that integrates Knowledge Distillation (KD) to circumvent the limitations of binarization. The system consists of three primary components:

  1. Receiver DNNs: Located at each user terminal, these networks estimate the channel from pilot signals and compress the information into a binary feedback vector.
  2. Transmitter DNN: Located at the BS, this network maps the aggregated binary feedback from all users to optimal precoding matrices.
  3. Auxiliary "Teacher" DNN: This network is used only during the training phase. It receives "lossless" floating-point signals from the receiver DNNs before the binarization layer.
The proposed deep-learning-based MU-MIMO system architecture, featuring receiver DNNs, a transmitter DNN, and an auxiliary transmitter DNN for knowledge distillation.

By providing a secondary path for gradients that bypasses the binarization layer, the auxiliary DNN guides the receiver DNNs toward better feature extraction. The actual binarization uses a Straight-Through Estimator (STE), but the KD process stabilizes the learning trajectory.

Experimental Setup

The framework was evaluated using synthetic Rayleigh fading channel data. The dataset comprised 100,000 training, validation, and test samples. The primary configuration involved a BS with $M=8$ transmit antennas serving users with $N=2$ receive antennas each. Performance was measured in terms of the total sum rate across varying Signal-to-Noise Ratios (SNR) and feedback bit budgets ($B$).

Key Results and Insights

The proposed KD-aided approach demonstrates significant performance gains over traditional linear precoding techniques such as Block Diagonalization (BD) and Regularized BD (RBD).

Sum rate performance comparison between the proposed KD-aided joint training, training without KD, and conventional baseline methods (BD and RBD).

Key findings include:

  • Superior Sum Rates: As shown in the performance curves, the KD-aided DNN consistently achieves higher sum rates than both conventional baselines and standard end-to-end training without distillation.
  • Effective Gradient Flow: KD acts as a regularizer, preventing the model from stagnating in poor local minima caused by the noisy gradients of the STE.
  • Non-Orthogonal Precoding: Interestingly, the DL-based system learns that non-orthogonal precoding columns can be more effective than strictly orthogonal ones when feedback is limited, a departure from traditional design principles.

Conclusion

This work highlights the potential of Knowledge Distillation to enable robust end-to-end learning in communication systems with discrete constraints. While the proposed method offers higher data rates, it comes with increased computational complexity at the receiver. Future research will focus on reducing this overhead and extending the model to handle dynamic user counts and more complex, spatially correlated channel environments.

Summary updated

Abstract

We propose a deep learning-based channel estimation, quantization, feedback, and precoding method for downlink multiuser multiple-input and multiple-output systems. In the proposed system, channel estimation and quantization for limited feedback are handled by a receiver deep neural network (DNN). Precoder selection is handled by a transmitter DNN. To emulate the traditional channel quantization, a binarization layer is adopted at each receiver DNN, and the binarization layer is also used to enable end-to-end learning. However, this can lead to inaccurate gradients, which can trap the receiver DNNs at a poor local minimum during training. To address this, we consider knowledge distillation, in which the existing DNNs are jointly trained with an auxiliary transmitter DNN. The use of an auxiliary DNN as a teacher network allows the receiver DNNs to additionally exploit lossless gradients, which is useful in avoiding a poor local minimum. For the same number of feedback bits, our DNN-based precoding scheme can achieve a higher downlink rate compared to conventional linear precoding with codebook-based limited feedback.
BibTeX
@article{kong2021knowledge,
  author = {Kyeongbo Kong and Woo‐Jin Song and Moonsik Min},
  title = {{Knowledge Distillation-aided End-to-End Learning for Linear Precoding in Multiuser MIMO Downlink Systems with Finite-Rate Feedback}},
  journal = {IEEE Trans. Vehicular Technology},
  year = {2021},
  doi = {10.1109/tvt.2021.3110608},
}
Figure