On the Capacity of a Class of Signal-Dependent Noise Channels

Hamid Ghourchian, Gholamali Aminian, Amin Gohari, Mahtab Mirmohseni, Masoumeh Nasiri-KenariIEEE Trans. Information Theory 2018 · DOI

TL;DR

This paper investigates the capacity of signal-dependent noise channels, a crucial model for systems like molecular and optical communication where noise variance changes with the signal itself. The authors introduce novel lower and upper bounds on channel capacity, providing new tools to analyze these complex systems and uncovering surprising behaviors, such as capacity not always increasing with reduced noise variance.

What Problem Does This Paper Address?

In many communication systems, the noise affecting the signal isn't constant. For instance, in molecular communication, the number of molecules detected (and thus the noise) can depend on the concentration of molecules released (the signal). This is known as signal-dependent noise. While the capacity of traditional additive white Gaussian noise (AWGN) channels is well-understood, analyzing channels where noise variance depends on the signal (Additive Signal-Dependent Noise, or ASDN channels) is significantly more challenging. Existing methods often struggle with the complexity of these channels, especially when dealing with general noise distributions and input constraints. This paper aims to provide a more comprehensive theoretical framework for understanding the capacity of these ASDN channels, including conditions for infinite capacity and the existence of optimal input distributions.

Methods

The authors employ a combination of information-theoretic tools and mathematical analysis. Key techniques include:

  • Majorization Inequality and Entropy Bounds: For lower bounds on capacity, they extend the majorization inequality and use entropy inequalities like $h(Y) > h(YZ)$ to derive bounds.
  • Symmetric Relative Entropy: For upper bounds, they leverage an idea based on symmetric relative entropy, particularly for the additive Gaussian signal-dependent noise (AGSDN) case.
  • Novel Lemma Development: A significant contribution is the development of new general lemmas (e.g., Lemma 1 and Lemma 3) that establish relationships between input and output entropies ($h(Y) \ge h(X)$) under specific conditions, which are crucial for deriving lower bounds.
  • Optimization Techniques: They utilize optimization techniques, including KKT conditions, to find optimal input distributions for maximizing mutual information under cost constraints.
  • Analysis of Special Cases: The paper analyzes specific forms of $\sigma(x)$ (the noise variance function) and noise distributions to illustrate their theoretical results and connect them to existing literature.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Non-Monotonic Capacity Behavior: Unlike AWGN channels where increasing noise variance always decreases capacity, signal-dependent noise channels can exhibit non-monotonic behavior. The paper shows that a channel with a larger noise variance function $\sigma_1(x)$ might have a higher capacity than a channel with a smaller $\sigma_2(x)$ under certain conditions.
  • Infinite Capacity Conditions: The paper identifies specific conditions under which the channel capacity can become infinite. This is particularly relevant for cases where the noise variance function $\sigma(x)$ approaches zero or infinity in certain ways.
  • Generalization of Existing Results: The work generalizes existing capacity results for specific AGSDN channels to a broader class of ASDN channels, making the theory more widely applicable.
  • New Tools for Analysis: The developed lemmas and bounding techniques provide new analytical tools for studying the capacity of complex, non-Gaussian, signal-dependent channels.

Key Results

  • Lower Bounds: Two novel lower bounds on the capacity of ASDN channels are derived. These bounds are applicable to a wide range of noise distributions and signal-dependent variance functions.
  • Upper Bound: An upper bound for AGSDN channels is provided, which is particularly suitable for low SNR regimes.
  • Finiteness and Infinity Conditions: Conditions are established for when the capacity of an ASDN channel is finite and when it can be infinite.
  • Existence of Capacity-Achieving Distributions: The paper proves the existence of capacity-achieving probability measures for finite capacity scenarios.
  • Numerical Examples: The paper includes numerical results and plots illustrating the tightness of the bounds and the behavior of capacity under different parameter settings, particularly for the AGSDN channel with $\sigma(x) = \sqrt{c_0 + c_1 x}$.

Limitations and Future Work

  • Assumptions on Noise: While the paper considers general ASDN channels, some results rely on specific properties of the noise distribution $Z$ (e.g., finite differential entropy, specific tail behaviors). Extending these results to more general noise distributions could be a future direction.
  • Complexity of Input Distributions: The paper provides bounds and conditions for capacity, but finding the exact capacity-achieving input distribution for general ASDN channels remains a challenging open problem.
  • Specific $\sigma(x)$ Forms: While the paper offers general results, specific analytical results are often derived for particular forms of $\sigma(x)$ (e.g., $\sqrt{c_0 + c_1 x}$). Further research could explore a wider range of $\sigma(x)$ functions.
  • Vector Channels: The paper primarily focuses on scalar channels. Extending the analysis to vector signal-dependent noise channels would be a natural next step.

Summary updated

BibTeX
@article{ghourchian2018capacity,
  author = {Hamid Ghourchian and Gholamali Aminian and Amin Gohari and Mahtab Mirmohseni and Masoumeh Nasiri-Kenari},
  title = {{On the Capacity of a Class of Signal-Dependent Noise Channels}},
  journal = {IEEE Trans. Information Theory},
  year = {2018},
  doi = {10.1109/tit.2018.2867599},
}
Figure