In introductory information theory, we are taught a fundamental elegance: for a channel with Additive White Gaussian Noise (AWGN) and an average power constraint, the capacity-achieving input distribution is a continuous Gaussian. This result is so pervasive that it often becomes our default intuition for all communication problems.
However, real-world hardware does not operate on average power alone. Transmitters have physical limits—voltages cannot exceed a certain threshold, and lasers cannot have negative intensity. When we replace the "soft" average power constraint with a "hard" peak power (amplitude) constraint, the elegant Gaussian curve vanishes. In its place, we find something surprising: the optimal way to communicate is through a finite set of discrete points.
The Geometry of Hard Limits
Why does a hard limit force discreteness? To understand this, consider the transition from an average power constraint to an amplitude constraint $A$. Under an average constraint, the signal can occasionally take very large values, provided they are rare enough to keep the mean square low. Under an amplitude constraint, the signal is strictly trapped in a box (or a sphere in higher dimensions).
As Alex R. Dytso, Semih Yagli, and Shlomo Shamai explain in The Capacity Achieving Distribution for the Amplitude Constrained Additive Gaussian Channel, the optimal distribution must balance two competing forces: spreading out to stay distinguishable amidst noise, and staying within the boundary. In a Gaussian distribution, the "tails" carry very little information but consume significant "space" near the boundaries. When the boundary is hard, the most efficient strategy is to "bunch up" the probability mass into discrete points (mass points) rather than spreading it thin.
In multi-dimensional vector channels, this geometry becomes even more structured. Alex R. Dytso, Mert Al, and Shlomo Shamai demonstrated in On the Capacity of the Peak Power Constrained Vector Gaussian Channel that these points arrange themselves into concentric spheres. For small amplitudes, a single shell is optimal; as the allowed power increases, the distribution "fractures" into multiple shells to fill the space efficiently.
How Many Points Are Enough?
If the optimal distribution is discrete, the next logical question is: how many points do we need? For decades, this was a mystery. Early proofs showed the number of points was finite but were non-constructive, leaving engineers to wonder if they needed ten points or ten thousand.
Recent breakthroughs have finally begun to pin down these numbers. In An Improved Lower Bound on Cardinality of Support of the Amplitude-Constrained AWGN Channel, Alex R. Dytso and Haiyang Wang disproved the long-standing conjecture that the number of points grows linearly with the amplitude. They proved that the number of points must grow at a "superlinear" rate of at least $A\sqrt{\log A}$.
This isn't just a theoretical curiosity. Knowing the "support size" (the number of discrete levels) tells us the fundamental complexity of the optimal transmitter. For example, in secure communications, Alex R. Dytso showed in Scalar Gaussian Wiretap Channel: Properties of the Support Size that the number of points needed to keep a message secret grows at most quadratically with power, ensuring that secure modulation schemes remain computationally feasible.
From Optical Fibers to Biological Neurons
The shift from continuous to discrete signaling appears across diverse physical systems whenever hard constraints are present:
- Optical Wireless: In systems like LiFi, signals are sent via light intensity, which cannot be negative. Anas Chaaban and Zouheir Rezki highlight in their tutorial on IM/DD channels that this non-negativity, combined with peak power limits, makes discrete signaling the mathematically optimal way to transmit data.
- Duty-Cycled Radios: When a radio is restricted to transmitting only a fraction of the time, Nevin L. Zhang and Dongning Guo found in Capacity of Gaussian Channels with Duty Cycle and Power Constraints that the optimal strategy is "on-off" signaling—a discrete choice of intensities rather than a continuous range.
- Neuroscience: Nature seems to have discovered this principle independently. Alexander P. Nikitin and colleagues showed in Neural Population Coding Is Optimized by Discrete Tuning Curves that to maximize information in the presence of Poisson noise, neurons should use step-like, discrete tuning curves. This explains why biological systems often categorize sensory inputs into distinct thresholds rather than maintaining a perfectly smooth analog representation.
Why It Works: The Intuition of Disturbance
A final piece of the puzzle comes from how we handle interference. In On Communication Through a Gaussian Channel With an MMSE Disturbance Constraint, Alex R. Dytso, Ronit Bustin, and others explore what happens when you want to send information without "disturbing" a neighbor too much.
They found that discrete signaling is superior to Gaussian signaling for limiting interference. Because a discrete constellation has "gaps" in its distribution, it effectively limits the amount of information (and thus the "disturbance") leaked to unintended receivers. This suggests that the discrete nature of optimal signaling is not just a consequence of power limits, but a fundamental property of efficient, constrained resource sharing in any noisy environment.