TL;DR
Vision models represent abstract concepts (like the number "3") as linear directions in their embedding space, but these directions are often "fragmented"—consistent within a specific visual style (e.g., handwritten digits) but different in another (e.g., hand gestures). As models scale and use more diverse training objectives (like CLIP), these fragmented directions converge into a single, globally consistent "concept vector," supporting the idea that AI is converging toward a shared, abstract model of reality.
What Problem Does This Paper Address?
The Linear Representation Hypothesis (LRH) suggests that high-level concepts are encoded as simple linear directions in a model's internal embedding space. While well-documented in Large Language Models (LLMs), vision presents a unique challenge: the same concept can look radically different depending on the context. For example, the concept of "three" can manifest as the digit "3," three fingers, or three dots. These "regimes" share almost nothing in common in pixel space.
The authors ask: Do vision models recognize the underlying abstraction? Does the vector for "adding one" to a set of dots look the same as the vector for "adding one" to a hand gesture? Understanding this is crucial for model interpretability and for building multimodal systems that truly "understand" the world rather than just memorizing visual patterns.
Methods
The researchers evaluated 46 different vision encoders (including CLIP, DINO, MAE, and supervised ResNets/ViTs) across 10 concept families. To measure linearity, they introduced several key frameworks:
- Regimes: They partitioned data into visually distinct groups (e.g., for the concept of "numerosity," regimes included MNIST digits, hand fingers, and rendered spheres).
- Approximate Counterfactuals: Unlike text, you can't easily swap one concept in an image while keeping everything else identical. The authors used linear sum assignment to find pairs of images within a regime that were as close as possible in embedding space but differed in the target concept (e.g., a "2" and a "3" with similar stroke thickness).
- Direction Consistency (DC): A metric that measures how well these counterfactual differences align with a single common direction. They distinguished between Local DC (consistency within one regime) and Cross-Regime DC (whether a direction from one regime works in another).
Datasets
The study utilized a diverse array of benchmarks to test 10 concept families:
- Numerosity: MNIST (digits), Fingers, CLEVR (objects), and a synthetic dot-counting dataset.
- Age: UTKFace and 11k-Hands.
- Object Identity: The PACS dataset (Photos, Art, Cartoons, Sketches).
- MIT-States Attributes: 7 binary attributes including Clean/Dirty, Dry/Wet, and Raw/Cooked.
Key Insights and Ideas
- Fragmented Linearity: This is the paper's central discovery. Many models exhibit strong "local" linearity (they have a clear vector for "3" within MNIST) but "fragmented" global linearity (the MNIST "3" vector doesn't work for the Fingers regime). As shown in Figure 1, a model can be locally linear but globally "lost."
- The Convergence Trend: There is a strong positive correlation ($\tau \in [0.38, 0.89]$) between local and global linearity. As models get better at representing a concept consistently in one context, they naturally begin to align those representations across all contexts.
- Platonic Convergence: The results support the "Platonic Representation Hypothesis"—the idea that as models become more powerful, their internal representations of the world converge toward a shared, objective ideal, regardless of their specific training data or architecture.
- Concrete vs. Abstract: Concepts with consistent visual cues (like "Age" via wrinkles or "Object Identity" via shape) show much higher global linearity than abstract concepts like "Numerosity" or "Cleanliness," where the visual manifestation varies wildly.
Key Results
- Local linearity is nearly universal: 99.1% of all (model, concept, regime) triplets showed statistically significant local linearity.
- Global linearity is the differentiator: Only 52.2% of tests passed for global linearity. Supervised models were the most fragmented, while CLIP-style models were "linear almost everywhere."
- CLIP vs. Supervised: As seen in Figure 2, a supervised ViT might have a fragmentation score of 0.67 for numerosity, while CLIP achieves a score of 0 (perfect global consistency).
- Linear Probing: In globally linear models, a concept direction discovered in one regime (e.g., "clean vs. dirty" in kitchens) can act as a highly accurate linear probe in a completely different regime (e.g., carpets), as shown in Figure 4.
Limitations and Future Work
The authors acknowledge that their counterfactual pairs are approximate, relying on embedding distance which might not perfectly isolate a single concept. Furthermore, the study is limited to 10 concept families; more complex or nuanced concepts might behave differently.
Future work could explore whether these linear directions can be used to "steer" vision models (e.g., making an image "cleaner" in representation space) and whether the emergence of global linearity is a prerequisite for the advanced reasoning seen in the latest multimodal models.
Summary updated