In most modern deep learning architectures, the way we aggregate information is mathematically "trapped." Whether it is a pooling layer in a CNN or a self-attention head in a Transformer, the output is almost always a weighted average of the inputs.
Geometrically, this means the output is restricted to the convex hull of the input features. If you imagine your input data points as vertices of a shape, the model can only "see" or "create" values that lie inside that shape. It can never extrapolate, never subtract, and never amplify signals beyond their original intensities.
Recent research suggests that this geometric restriction—often called the "probability cage"—is a fundamental bottleneck for model expressivity, preventing networks from learning complex logic or reaching high correlation in regression tasks.
The Geometry of the Probability Cage
In standard attention, the softmax function ensures that attention weights are non-negative and sum to one. As Oliver Richter and Roger Wattenhofer point out in Normalized Attention Without Probability Cage, this forces the output to be a convex combination of input vectors.
This is a problem for two main reasons:
- Logical Limitations: A single attention head trapped in a convex hull cannot represent simple non-linear functions like XOR. To compute such logic, the model needs the freedom to move outside the boundaries defined by its inputs.
- The Extrapolation Ceiling: In regression tasks, Jingquan Yan and colleagues identify a "PCC Plateau". Because the model is a "convex aggregator," it cannot generate outputs that go beyond the range of its immediate features. This creates a capacity ceiling when the model needs to predict extreme values or handle highly similar (homogeneous) data.
Subtractive Attention and Negative Weights
If the convex hull is a cage, the bars are the non-negativity and sum-to-one constraints. By relaxing these, we allow for "subtractive" operations.
In Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models, Zichen Miao and Qiang Qiu propose using unconstrained coefficients to mix attention heads. By allowing negative coefficients, the model can perform subtractive attention—effectively suppressing specific noise or irrelevant information rather than just deciding how much of each "positive" signal to keep. This expands the representational space far beyond the original geometric boundary of the input embeddings.
Beyond Simple Averages in Pooling
This constraint isn't limited to Transformers; it has long existed in Convolutional Neural Networks (CNNs) through max and average pooling. Traditional pooling operations are mathematically restricted to the range of their local neighborhood.
Takumi Kobayashi addressed this in Gaussian-Based Pooling for Convolutional Neural Networks. Instead of simply picking the maximum value or the average, Kobayashi models local activations using Gaussian statistics (mean and standard deviation). This probabilistic framework allows the network to amplify or transform signals during downsampling, providing a more flexible and discriminative feature extraction than "convex" methods allow.
Factorization as a Way Out
Another way to break the bottleneck is to rethink the mixing matrix entirely. Standard dot-product attention is often low-rank, which further limits what the model can represent.
In Paramixer: Parameterizing Mixing Links in Sparse Factors Works Better than Dot-Product Self-Attention, Tong Yu and Zhirong Yang replace the softmax-based mixing with sparse, full-rank components parameterized by MLPs. By moving away from the softmax "mixing link," Paramixer bypasses the low-rank bottleneck and the convex hull constraint simultaneously. This allows the model to synthesize information more flexibly, which proves particularly effective for extremely long sequences where traditional attention's mathematical restrictions become most apparent.
Summary of the Shift
The transition across these papers represents a move from probabilistic aggregation to linear transformation.
| Feature | Traditional (Convex) | Modern (Non-Convex) |
|---|---|---|
| Mechanism | Softmax, Max/Avg Pooling | Normalized Linear, Gaussian Pooling |
| Constraint | $w_i \ge 0, \sum w_i = 1$ | Unconstrained or Gaussian-parameterized |
| Geometry | Inside the Convex Hull | Anywhere in the Feature Space |
| Capability | Interpolation | Extrapolation, XOR logic, Suppression |
By escaping the convex hull, these models gain the ability to generate novel feature representations that were previously mathematically impossible to reach, leading to better performance in everything from genomic modeling to high-precision regression.