For decades, the dominant theory of communication efficiency has been the Uniform Information Density (UID) hypothesis. It suggests that to communicate effectively over a noisy channel, humans should spread information evenly across a sequence of words, avoiding sudden spikes that might overwhelm a listener.
However, recent research suggests that "perfect" uniformity is actually a hallmark of machines, not humans. While Large Language Models (LLMs) tend to produce flat, predictable probability curves, human language is characterized by a distinct "rhythm" of information—a series of periodic peaks and troughs in surprisal that align with the structural building blocks of discourse.
The Harmonic Structure of Surprisal
If you measure the "surprisal" (the mathematical unpredictability) of each word in a human-written essay, you won't find a straight line. Instead, you'll find a wave.
Zuhao Yang and colleagues in FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy discovered that human English text exhibits a consistent periodic rhythm, with information density peaking and troughing approximately every eight tokens. By treating these surprisal sequences as digital signals and applying Fourier Transforms, they found a "temporal signature" in human writing that persists across genres, from creative stories to journalistic reports.
This isn't just random noise. In The Harmonic Structure of Information Contours, Eleftheria Tsipidi and Ethan Wilcox argue that these fluctuations are synchronized with linguistic boundaries. Surprisal levels rise and fall in predictable waves that correspond to the length of sentences and paragraphs, creating what they call "information arcs."
Why Humans "Wiggle"
Why don't we just speak at a constant rate? The Structured Context Hypothesis, proposed by Tsipidi et al. in Surprise! Uniform Information Density Isn't the Whole Story, suggests these "wiggles" are a byproduct of how we organize thoughts.
Information density tends to change monotonically based on a word's position within a discourse unit. As we move through a paragraph or a rhetorical argument, we adjust the density of new information to manage the listener's attention. We might start with routine, low-surprisal "anchoring" language before building to a creative, high-surprisal point. This creates a "bursty" pattern—segments of routine language interspersed with bursts of creativity.
The AI "Flatness" Problem
Current AI models are often too efficient. Because they are trained to minimize loss across vast datasets, they often default to a strategy that adheres strictly to UID.
Advik Raj Basani and Pin-Yu Chen leverage this discrepancy in Diversity Boosts AI-Generated Text Detection. Their framework, DivEye, identifies synthetic text by looking for a lack of "rhythmic unpredictability." While humans are stylistically diverse and non-stationary, LLMs produce unnaturally flat probability curves.
This flatness may also hinder learning. Zhihao Zhang and colleagues found in Cross-domain NER with Generated Task-Oriented Knowledge that unstructured training data often suffers from inconsistent information density, making it harder for models to internalize patterns. By using LLMs to generate structured "reasoning chains" with more intentional information flow, they could significantly improve model performance on complex tasks.
The Alignment Trade-off
If human language is rhythmic and bursty, why don't we just fine-tune models to act that way? It turns out there is a cognitive cost.
In Reverse-Engineering the Reader, Samuel Kiegeland and Ryan Cotterell attempted to align LLMs so their internal surprisal matched human reading times. While this successfully "reverse-engineered" a more human-like cognitive model—complete with higher variance in surprisal—the model's performance on standard NLP tasks plummeted.
This suggests a fundamental tension: the "smooth" information flow that makes a model a great next-word predictor is fundamentally different from the "rhythmic" flow that makes a speaker sound human. Understanding this gap is key to moving beyond models that merely predict text, toward models that communicate with the natural cadence of human thought.