Knowledge Distillation-Aided End-to-End Learning for Linear Precoding in Multiuser MIMO Downlink Systems With Finite-Rate Feedback

Kyeongbo Kong, Woo-Jin Song, Moonsik Min

TL;DR

This paper introduces an end-to-end deep learning framework to optimize channel estimation and precoding in multiuser MIMO systems. By using Knowledge Distillation (KD), the authors overcome the "vanishing gradient" problem caused by binarizing feedback data, allowing the system to achieve higher data rates than traditional codebook-based methods.

What Problem Does This Paper Address?

In Multiuser Multiple-Input Multiple-Output (MU-MIMO) systems, a base station (BS) sends data to multiple users simultaneously. To do this effectively without causing massive interference, the BS needs to know the "shape" of the wireless channel for each user—this is called Channel State Information (CSI).

In systems using Frequency-Division Duplexing (FDD), users must estimate their own channels and send that info back to the BS. Because uplink bandwidth is precious, this feedback is limited to a small number of bits (finite-rate feedback). Traditionally, this is done using pre-defined "codebooks."

While Deep Learning (DL) can theoretically outperform these manual designs, there is a major hurdle: Binarization. To send feedback as bits, the neural network must use a step function (like a sign function), which has a gradient of zero almost everywhere. This "breaks" the backpropagation process, often trapping the model in poor local minima during training.

Methods

The authors propose a joint training architecture consisting of three main parts:

  1. Receiver DNNs: Located at each user's device, these networks take raw pilot signals, estimate the channel, and compress that info into a binary vector (the feedback bits).
  2. Transmitter DNN: Located at the Base Station, this network takes the binary feedback from all users and calculates the optimal precoding matrices to maximize the total sum rate.
  3. The Auxiliary "Teacher" DNN: This is the core innovation. During training, an auxiliary transmitter network is connected to the receiver DNNs before the binarization layer.

Because the teacher network receives "lossless" floating-point gradients rather than noisy, binarized ones, it can effectively guide the receiver DNNs toward better feature extraction. The system uses a Straight-Through Estimator (STE) for the actual binarization layer but relies on the KD process to stabilize and improve the learning direction.

Datasets

The study uses synthetically generated channel data.

  • Channel Model: Independent and identically distributed (i.i.d.) complex Gaussian random variables (Rayleigh fading).
  • Scale: The authors evaluated the system using 100,000 fixed test datasets and 100,000 validation datasets.
  • Configurations: They primarily tested a system with $M=8$ transmit antennas and $N=2$ receive antennas per user, varying the number of users ($K$) and feedback bits ($B$).

Key Insights and Ideas

  • KD as a Gradient Fix: Usually, Knowledge Distillation is used to shrink large models into small ones. Here, it is cleverly used as a bypass for non-differentiable layers, providing "lossless gradients" to parts of the network that would otherwise be "blinded" by binarization.
  • Moving Beyond Orthogonality: Traditional linear precoding (like Block Diagonalization) often forces precoding columns to be orthogonal to minimize interference. The DL-based approach discovers that with limited feedback, non-orthogonal columns can actually achieve higher total sum rates.
  • Robustness to STE Noise: While STEs are common in quantized neural networks, they introduce noise. The joint training with KD acts as a regularizer that prevents the model from getting stuck in the "poor local minima" that typically plague STE-based training.

Key Results

  • Higher Sum Rates: The proposed KD-aided DNN consistently outperformed conventional Block Diagonalization (BD) and Regularized BD (RBD) across various Signal-to-Noise Ratios (SNR).
  • Efficiency: For the same number of feedback bits, the DL method achieved a significantly higher downlink rate. For example, in a scenario with 24 total feedback bits, the DL approach maintained a steady lead over traditional RBD as the number of receive antennas increased.
  • Stable Convergence: Training curves showed that the KD-aided method converged much more reliably to a high-performance solution compared to standard end-to-end training without distillation.

Limitations and Future Work

  • Computational Overhead: The authors note that the receiver DNNs require more Floating-Point Operations (FLOPs) than traditional methods (e.g., ~471k vs ~310k in one test case). This might be a concern for low-power mobile devices.
  • Retraining Requirements: The current model needs to be retrained if the number of users ($K$) changes. A more flexible architecture that handles a dynamic number of users would be a significant next step.
  • Channel Realism: The study uses i.i.d. Gaussian channels. Testing on more complex, spatially correlated, or measured "real-world" channel models would further validate the approach for 5G/6G deployment.

Summary updated

Abstract

We propose a deep learning-based channel estimation, quantization, feedback, and precoding method for downlink multiuser multiple-input and multiple-output systems. In the proposed system, channel estimation and quantization for limited feedback are handled by a receiver deep neural network (DNN). Precoder selection is handled by a transmitter DNN. To emulate the traditional channel quantization, a binarization layer is adopted at each receiver DNN, and the binarization layer is also used to enable end-to-end learning. However, this can lead to inaccurate gradients, which can trap the receiver DNNs at a poor local minimum during training. To address this, we consider knowledge distillation, in which the existing DNNs are jointly trained with an auxiliary transmitter DNN. The use of an auxiliary DNN as a teacher network allows the receiver DNNs to additionally exploit lossless gradients, which is useful in avoiding a poor local minimum. For the same number of feedback bits, our DNN-based precoding scheme can achieve a higher downlink rate compared to conventional linear precoding with codebook-based limited feedback.
BibTeX
@article{kong2021knowledge,
  author = {Kyeongbo Kong and Woo-Jin Song and Moonsik Min},
  title = {{Knowledge Distillation-Aided End-to-End Learning for Linear Precoding in Multiuser MIMO Downlink Systems With Finite-Rate Feedback}},
  journal = {IEEE Trans. Vehicular Technology},
  year = {2021},
  doi = {10.1109/tvt.2021.3110608},
}
Figure