In classical information theory, the Gaussian distribution is often celebrated as the ideal way to transmit information. For a standard additive white Gaussian noise (AWGN) channel with an average power constraint, a continuous Gaussian input distribution maximizes the data rate. However, this optimality is fragile. When physical systems face real-world constraints—such as peak power limits, low-resolution hardware, or biological stochasticity—the most efficient way to communicate is not through a smooth range of values, but through a discrete set of specific signal levels.
The Optimality of Discreteness
The shift from continuous to discrete signaling occurs because constraints often restrict the "space" available for the signal. When a transmitter cannot exceed a certain peak amplitude, spreading the signal across a continuous range becomes less efficient than concentrating it into a few well-separated "mass points."
This phenomenon is observed across diverse fields. In biological systems, Nikitin et al. demonstrate that to maximize information transmission in noisy neurons, the optimal response function is a discrete, step-like tuning curve. By quantizing the input signal into distinct firing rate levels, the system minimizes the impact of Poisson noise. Similarly, in molecular communication, where information is carried by chemical concentrations, Tahmasbi and Fekri prove that the maximum information rate is achievable using a finite set of concentration levels rather than a continuous range.
Hard Constraints and Phase Transitions
The transition from continuous to discrete signaling is often driven by the nature of the cost function or the physical limits of the hardware. Stapmanns et al. identify a "phase transition" in channels with uniform noise: the optimal input is discrete when the cost function is concave but becomes continuous the moment it becomes convex.
Peak power (amplitude) constraints are a primary driver of this behavior:
- Secrecy and Security: For wiretap channels—where a message must be kept secret from an eavesdropper—Dytso and Dytso show that the secrecy-capacity-achieving input is discrete with a finite number of support points. Zorgui et al. further establish that for secret-key agreement under peak power constraints, continuous Gaussian signaling is outperformed by discrete mass points.
- Energy Harvesting: In systems that rely on ambient energy, Erkip et al. note that discrete signaling is more efficient than continuous signaling when transmitters face instantaneous energy constraints without battery storage.
- Duty Cycles: When a radio can only transmit for a fraction of the time, Guo demonstrates that the capacity-achieving distribution is discrete, strategically using "off" periods as a zero-symbol to convey information.
Hardware-Driven Quantization
Modern wireless systems often use simplified hardware to reduce power consumption, which fundamentally changes the optimal signaling strategy. A prominent example is the use of low-resolution analog-to-digital converters (ADCs).
Rassouli et al. prove that in a Gaussian Multiple Access Channel where the receiver uses a one-bit ADC, the optimal communication strategy requires discrete input signals. In these scenarios, the receiver's inability to distinguish between fine-grained continuous values makes continuous signaling wasteful.
This principle extends to complex network architectures. In Reconfigurable Intelligent Surface (RIS) assisted symbiotic radios, Zhang et al. find that while a continuous Rayleigh distribution might maximize the sum rate, the optimal distributions for other points on the capacity boundary are frequently discrete.
Implications for System Design
Recognizing the optimality of discrete signaling allows for more efficient practical designs. For instance, in overloaded CDMA systems where users exceed bandwidth, Khoozani et al. show that signature sequences optimized for Gaussian inputs underperform; instead, matrices must be tailored to the discrete binary nature of the signals.
Furthermore, in noncoherent fading channels—where neither side knows the channel state—Gursoy and Verdú prove that using a finite set of specific amplitude levels is more efficient than a continuous range. By focusing on a discrete set of mass points, designers can achieve higher rates and lower complexity in environments where traditional Gaussian assumptions fail.
Key Foundational Work
The Noncoherent Rician Fading Channel - Part I : Structure of the Capacity-Achieving Input (2005) — contributed 1 concept elements to this direction.
Neural Population Coding Is Optimized by Discrete Tuning Curves (2009) — contributed 1 concept elements to this direction.