Learning Channel Capacity with Neural Mutual Information Estimator Based on Message Importance Measure

Zhefan Li, Rui She, Pingyi Fan, Chenghui Peng, Khaled B. Letaief

Abstract

Channel capacity estimation plays a crucial role in beyond 5G intelligent communications. Despite its significance, this task is challenging for a majority of channels, especially for the complex channels not modeled as the well-known typical ones. Recently, neural networks have been used in mutual information estimation and optimization. They are particularly considered as efficient tools for learning channel capacity. In this paper, we propose a cooperative framework to simultaneously estimate channel capacity and design the optimal codebook. First, we will leverage MIM-based GAN, a novel form of generative adversarial network (GAN) using message importance measure (MIM) as the information distance, into mutual information estimation, and develop a novel method, named MIM-based mutual information estimator (MMIE). Then, we design a generalized cooperative framework for channel capacity learning, in which a generator is regarded as an encoder producing the channel input, while a discriminator is the mutual information estimator that assesses the performance of the generator. Through the adversarial training, the generator automatically learns the optimal codebook and the discriminator estimates the channel capacity. Numerical experiments will demonstrate that compared with several conventional estimators, the MMIE achieves state-of-the-art performance in terms of accuracy and stability.

Introduction

In the quest for 6G and beyond, understanding channel capacity—the theoretical speed limit of a communication link—is vital. While elegant formulas exist for simple scenarios like Additive White Gaussian Noise (AWGN), real-world channels are often "black boxes" with complex, non-linear behaviors that lack closed-form solutions.

Existing approaches to this problem generally fall into two categories: direct optimization and two-step optimization (using either traditional or neural methods). As summarized in Figure 1, traditional numerical methods often struggle with continuous inputs and high-dimensional data, while early neural estimators can be unstable.

Comparison of mutual information estimation approaches, highlighting the advantages and disadvantages of current methods.

Specifically, neural estimators like MINE or iDIME, while powerful, often suffer from high variance or "numerical explosions" (outputting infinity) when the Signal-to-Noise Ratio (SNR) is high. This makes them unreliable for practical engineering. This paper introduces a novel estimator, MMIE, designed to be both mathematically grounded and numerically robust across diverse channel conditions.

Methodology

The core innovation of this work is the integration of the Message Importance Measure (MIM). Unlike standard Shannon entropy, which relies on logarithmic functions, MIM utilizes exponential functions to characterize information. This shift is significant because exponential functions are more effective at emphasizing "rare events" and provide enhanced stability during neural network training.

MMIE and $\alpha$-MMIE

The proposed MIM-based mutual information estimator (MMIE) is derived from the objective function of an MIM-based Generative Adversarial Network (GAN). To further improve performance, an $\alpha$-parameterized version ($\alpha$-MMIE) was developed. This allows for "centering" the discriminator's output, effectively preventing the vanishing gradient problems that often plague other neural estimators at high SNRs.

Cooperative Framework

The system employs a GAN-like cooperative architecture:

  • Generator (Encoder): Responsible for creating the channel input or codebook.
  • Discriminator (MMIE): Acts as the mutual information estimator to assess the generator's performance.

These components are trained together in an adversarial manner: the discriminator strives to accurately estimate the information flow, while the generator attempts to maximize it.

Results and Discussion

The framework was evaluated using simulated communication environments, including 2D and 10D AWGN channels and discrete message sets.

Stability and Accuracy

A major finding is the superior stability of MMIE. In 10-dimensional tests at 15dB SNR, the popular dDIME estimator failed 100% of the time due to numerical issues (e.g., dividing by zero). In contrast, MMIE and $\alpha$-MMIE maintained a 0% failure rate. Furthermore, MMIE consistently demonstrated lower Root Mean Square Error (RMSE) and lower bias compared to traditional estimators like KSG and other neural methods like MINE.

Codebook Optimization

Because the framework is cooperative, the generator essentially "learns" the optimal way to shape signals for a specific channel. Results showed that the codebooks generated by the MMIE framework outperformed standard industry benchmarks like 8-PSK (Phase Shift Keying) constellations, proving that the learned signal shapes are more efficient at navigating channel noise.

Conclusion

The MMIE framework provides a robust and accurate solution for learning channel capacity and designing optimal codebooks simultaneously. By replacing logarithmic losses with exponential-based MIM losses, the system achieves unprecedented stability at high SNRs. Future work will focus on extending this framework to channels with memory and feedback loops, as well as validating performance on actual radio hardware.

Summary updated

Abstract

Channel capacity estimation plays a crucial role in beyond 5G intelligent communications. Despite its significance, this task is challenging for a majority of channels, especially for the complex channels not modeled as the well-known typical ones. Recently, neural networks have been used in mutual information estimation and optimization. They are particularly considered as efficient tools for learning channel capacity. In this paper, we propose a cooperative framework to simultaneously estimate channel capacity and design the optimal codebook. First, we will leverage MIM-based GAN, a novel form of generative adversarial network (GAN) using message importance measure (MIM) as the information distance, into mutual information estimation, and develop a novel method, named MIM-based mutual information estimator (MMIE). Then, we design a generalized cooperative framework for channel capacity learning, in which a generator is regarded as an encoder producing the channel input, while a discriminator is the mutual information estimator that assesses the performance of the generator. Through the adversarial training, the generator automatically learns the optimal codebook and the discriminator estimates the channel capacity. Numerical experiments will demonstrate that compared with several conventional estimators, the MMIE achieves state-of-the-art performance in terms of accuracy and stability.
BibTeX
@article{li2024learning,
  author = {Zhefan Li and Rui She and Pingyi Fan and Chenghui Peng and Khaled B. Letaief},
  title = {{Learning Channel Capacity with Neural Mutual Information Estimator Based on Message Importance Measure}},
  journal = {IEEE Trans. Communications},
  year = {2024},
  url = {https://openreview.net/forum?id=B18I15y9IW},
}
Figure