When we inspect the internal activations of a deep neural network, we often expect to find clean, human-understandable concept detectors. We hope to find a "cat neuron," a "legal text neuron," or an "addition operator neuron." Instead, empirical investigations frequently reveal a messy reality: individual neurons are notoriously polysemantic, lighting up for a bizarre, seemingly unrelated collection of concepts. A single neuron might activate for a optical illusion, a mathematical equation, and a specific German verb.
Early interpretability frameworks, such as the network dissection method proposed by Bau et al., demonstrated that while some neurons align beautifully with discrete semantic concepts, a significant portion of a network's internal units remain stubbornly polysemantic. As highlighted in the comprehensive survey by Räuker et al., this lack of a simple one-to-one mapping between neurons and human concepts is not a minor glitch—it is a fundamental property of deep neural networks that complicates our ability to decode their internal logic.
To truly understand why networks behave this way, we have to look at the geometry of high-dimensional spaces.
The Geometry of Superposition
Why do neural networks choose to entangle their representations rather than dedicating one neuron per concept? The answer lies in a phenomenon known as feature superposition.
Imagine a network that needs to represent a vast dictionary of real-world features using a limited number of hidden dimensions (neurons). If the network forces each feature to align perfectly with a specific neuron axis, it can only learn as many features as it has dimensions. This is a severe bottleneck for models operating on web-scale data.
As Bereska et al. explain, the superposition hypothesis suggests that models overcome this dimensional constraint by compressing more features into a hidden space than they have physical neurons. This compression relies on a remarkable property of high-dimensional geometry: in a space with a large number of dimensions, you can choose an exponential number of vectors that are nearly orthogonal to one another.
To build intuition, consider the difference between low and high dimensions:
- In a 2D space, you can only find 2 perfectly orthogonal (90-degree) vectors. If you try to pack 10 vectors into a 2D space, they will heavily overlap and interfere with each other.
- In a 10,000-dimensional space, however, you can choose millions of vectors whose pairwise dot products are incredibly close to zero.
By utilizing these nearly orthogonal directions as the "axes" for different features, the network can represent a massive feature set while tolerating only a tiny amount of mutual interference. Yizhou Liu et al. demonstrate that this geometric strategy is highly efficient, showing that representation superposition is a primary driver behind the robust power-law scaling behaviors observed in large language models. The network trades off absolute conceptual purity for an exponential increase in representational capacity.
The Cost of Entanglement
While superposition provides a massive boost to a model's capacity, it introduces a severe engineering challenge: feature entanglement. Because features are not aligned with individual neurons, the standard basis vectors of the network's latent space become polysemantic.
This entanglement creates a major barrier for any task that requires precise control over a model's internal state, such as model editing, safety alignment, or knowledge unlearning.
Standard Basis (Neurons)
│
├── Neuron 1 ──► Activates for: [Concept A] + [Concept B] + [Concept C] (Polysemantic)
└── Neuron 2 ──► Activates for: [Concept B] + [Concept D] + [Concept E] (Polysemantic)
Decompressed Basis (Sparse Autoencoder)
│
├── Feature 1 ──► Activates strictly for: [Concept A] (Monosemantic)
├── Feature 2 ──► Activates strictly for: [Concept B] (Monosemantic)
└── Feature 3 ──► Activates strictly for: [Concept C] (Monosemantic)
1. Collateral Damage in Model Editing and Unlearning
When we attempt to modify or erase a specific piece of knowledge, we run directly into the wall of superposition. Hong et al. discovered that standard unlearning techniques often fail to permanently erase information because the underlying knowledge is distributed across polysemantic vectors.
Similarly, Singh et al. observed that attempting to suppress a specific error by deleting or scaling down attention heads is often ineffective. Because these internal components are polysemantic—performing multiple functions simultaneously—modifying them to fix a single behavior inadvertently degrades the model's general linguistic fluency or reasoning capabilities.
2. Safety and Utility Conflicts
This entanglement also explains why safety tuning can sometimes degrade a model's capabilities. Dongrui Liu et al. found a high degree of structural overlap between the neurons responsible for safety guardrails and those responsible for general utility. When parameters are modified during fine-tuning or model merging, updates intended to optimize one capability can blindly overwrite or disrupt the logic governing another.
Untangling the Web with Dictionary Learning
If the network's native neurons are fundamentally polysemantic, how can we ever hope to understand or steer them? The solution is to find a new coordinate system where the features are axis-aligned. This is achieved through dictionary learning, typically implemented via Sparse Autoencoders (SAEs).
The core idea behind an SAE is to take the dense, polysemantic activations of a model layer and project them into a much higher-dimensional, sparse categorical space. By enforcing a strict sparsity constraint (forcing the autoencoder to use only a few active dimensions at a time), we incentivize the SAE to find the true, underlying directions of the features in superposition.
[Dense Activation Vector] ──► [Encoder (Weight Matrix)] ──► [High-Dimensional Sparse Layer]
(Polysemantic) (Monosemantic Features)
When successful, this process yields monosemantic features—latents that activate for exactly one human-understandable concept. For instance, Parekh et al. utilized dictionary learning to decompose the internal representations of Large Multimodal Models, successfully isolating distinct visual and textual concepts from a single entangled token vector.
Architectural Refinements
Standard SAEs suffer from a practical limitation known as "shrinkage," where the regularization used to enforce sparsity systematically underestimates feature magnitudes. To solve this, Rajamanoharan introduced the Gated Sparse Autoencoder, which decouples feature detection from magnitude estimation. This architectural shift allows for cleaner feature extraction without sacrificing reconstruction fidelity.
Alternative approaches seek to bypass continuous representations entirely. Tamkin introduced codebook features, which refactor neural networks into discrete states using vector quantization bottlenecks. By forcing the network to choose from a discrete set of codes during training, the model is prevented from falling into dense superposition, making its internal states natively steerable.
The Frontier of Residual Polysemanticity
While dictionary learning has dramatically advanced our understanding of neural networks, achieving perfect monosemanticity remains an open research challenge. Even when using massive sparse autoencoders, researchers frequently encounter residual polysemanticity.
As observed by Joseph et al. when steering vision transformers, some extracted features continue to respond to multiple unrelated concepts. This suggests that feature superposition is not just a flat linear puzzle to be solved with a single matrix inversion, but a complex, multi-layered compression strategy that deep networks leverage across their entire depth. Understanding how to completely untangle these representations is one of the most critical frontiers in mechanistic interpretability today.