Abstract
Large Language Model (LLM) embeddings are typically assumed to lie on a single, smooth global manifold. This paper argues that they actually form a "stratified space" composed of multiple sub-manifolds of varying dimensions. We introduce a novel unsupervised Mixture-of-Experts (MoE) framework using dictionary learning to cluster and analyze frozen LLM embeddings. Our findings reveal that LLM latent spaces are highly non-uniform, with different data domains naturally routing to distinct geometric strata, offering new insights into model interpretability and latent space geometry.
Introduction
LLMs generate rich, high-dimensional embeddings that capture deep semantic relationships. When analyzing these latent spaces, researchers often assume the data lies on a single, continuous global manifold. However, real-world data is heterogeneous—different domains, such as short tweets versus long news articles, possess vastly different complexities and perplexities.
If the embedding space is actually a stratified space—a union of multiple lower-dimensional local manifolds (strata) rather than one smooth surface—then treating it as a single manifold limits our ability to understand how LLMs organize concepts. This work proposes a framework to explicitly test for, discover, and analyze these localized, stratified structures within the embedding spaces of popular LLMs.
Methodology
Instead of using MoE to scale model capacity during training, we repurpose the MoE architecture as an unsupervised analysis tool applied to fixed LLM embeddings (specifically, the last hidden state).
Dictionary Learning Experts
The "experts" in this MoE are Dictionary Learning algorithms implemented via Learnable ISTA (LISTA). Crucially, each expert is hard-coded with a different sparsity level. This allows the model to associate different geometric complexities with different experts.
Attention-Based Gating
An attention-based soft-gating network routes each embedding to the most suitable expert(s). It projects the input into a query space, compares it against learnable key vectors for each expert, and outputs routing probabilities via softmax. By observing which sparsity-constrained expert an embedding is routed to, the framework effectively partitions the embedding space into specialized sub-manifolds (strata).
Experimental Setup
To test how different domains behave in the latent space, we utilized a custom multi-domain dataset including IMDB, Rotten Tomatoes, Amazon Polarity, GLUE/SST2, TweetEval, and AG News. We evaluated embeddings generated by a range of models, including BERT, RoBERTa, GPT-3, Llama-3.2-1B, and DeepSeek-R1-Distill-Qwen-1.5B.
Key Results and Discussion
Our analysis confirms that the embedding space is definitively non-uniform, consisting of varying intrinsic dimensions and complexities depending on the data domain.
Stratification and Model Scale
Larger models (like Llama-3 and DeepSeek) exhibit a much clearer separation of domains into distinct strata compared to smaller models like BERT. Bigger models also show highly concentrated expert selection, implying their embedding spaces have more uniform perplexity within specific domains. Conversely, smaller models spread assignments across experts, suggesting a "rougher" latent space.
Semantic Alignment
Mathematically derived strata naturally align with human-understandable semantic domains. While the MoE model clustered the geometry blindly, those geometric clusters mapped cleanly back to the original dataset sources. Using PCA to capture 75% of the data variance within each learned stratum, we proved that dimensionality fluctuates significantly (e.g., in Llama-3.2-1B, intrinsic dimensions ranged from 2 to 16 across identified strata).
Conclusion
This work provides empirical evidence that LLM embedding spaces are stratified manifolds. While the current findings are strong, future work will focus on rigorous mathematical and topological proofs of these structures and exploring how stratification can be leveraged to improve LLM performance, architectural design, and interpretability.
Summary updated