Towards Atoms of Large Language Models

Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

TL;DR

This paper introduces Atom Theory, a rigorous framework to define, evaluate, and identify the fundamental representational units ("atoms") of Large Language Models (LLMs). By replacing standard Euclidean metrics with a novel non-Euclidean Atomic Inner Product (AIP), the authors correct a pervasive geometric bias in LLM representations. Using Threshold-Activated Sparse Autoencoders (TSAEs), they successfully extract representational units across Gemma-2 and Llama-3.1 models that achieve near-perfect faithfulness ($R^2 \ge 99.9\%$) and structural stability ($q^* \ge 99.8\%$).

What Problem Does This Paper Address?

Mechanistic interpretability aims to reverse-engineer LLMs by finding their basic building blocks—termed Fundamental Representational Units (FRUs). Historically, researchers looked at individual neurons, but neurons are notoriously polysemantic (they activate for many unrelated concepts). More recently, researchers have used Sparse Autoencoders (SAEs) to extract features (linear directions in activation space). However, features suffer from their own issues: they are often unfaithful (failing to fully reconstruct the original activations) and unstable (splitting or merging depending on the dictionary size).

The root of the problem is that there has been no formal definition of what an ideal FRU should be, nor a canonical geometry to measure them. LLM activation spaces suffer from a severe "representation shift"—an anisotropic distortion caused by the Softmax operation that pushes the centroid of activation angles away from 90 degrees, making standard Euclidean inner products misleading.

Methods

The authors propose Atom Theory to formalize the geometry and properties of ideal representational units:

  1. Atomic Inner Product (AIP): A non-Euclidean inner product designed to counteract the representation shift. It normalizes the space such that independent concepts are truly orthogonal (90 degrees apart), providing a canonical geometry for internal representations.
  2. Formal Definition of Atoms: An ideal "atom set" must satisfy three core properties under the AIP:
  3. Representability: Activations can be faithfully reconstructed.
  4. Sparsity: Each activation map utilizes only a few atoms.
  5. Separability: Atoms are approximately orthogonal to one another.
  6. Evaluation Metrics: The authors introduce two quantitative metrics:
  7. Faithfulness ($R^2$): Measures reconstruction fidelity.
  8. Stability ($q^*$): A metric derived from compressed sensing that quantifies "monorepresentationality"—ensuring that combinations of atoms remain unique and distinguishable.
  9. Threshold-Activated SAEs (TSAEs): The authors mathematically prove that standard ReLU-based SAEs are theoretically insufficient for atom identification because they lack a hard thresholding mechanism. Instead, they utilize TSAEs (specifically using JumpReLU activation functions) to reliably detect and recover the underlying atom set.
Figure 1. Illustration of Atom Theory. (a) Atoms are defined based on the atomic inner product, inducing representability, sparsity, and separability. (b) Atoms are evaluated by faithfulness (R2) and stability (q∗), measuring fidelity and stable-atom fraction. (c) Threshold-activated SAEs enable atom identification, with the encoder as an atom detector and the decoder as the target atom set.

Datasets

The experiments utilize several distinct datasets to extract and evaluate activations:

  • WikiData & CounterFact: Used to extract entity-specific knowledge activations across model layers (referred to as the "knowledge atomization" task).
  • Wikipedia Corpora: Used as a baseline for general natural language text activations and to estimate the AIP matrix.
  • MATH500: A dataset of complex mathematical reasoning problems used for task-level ablation studies.

Key Insights and Ideas

  • The Euclidean Metric is Flawed for LLMs: Under standard Euclidean inner products, unrelated concepts show high cosine similarity simply due to a global coordinate bias. The AIP successfully removes this global bias, restoring the angular distribution centroid to 90 degrees across multiple model families.
  • Neurons and Features are Not Ideal Atoms: Neurons have perfect faithfulness ($R^2=1$) but terrible stability ($q^*=0.5\%$). Conversely, standard features from open-source repositories like GemmaScope improve stability ($q^*=68.2\%$) but sacrifice faithfulness ($R^2=48.8\%$). Ideal atoms must maximize both.
  • Capacity Must Meet Data Scale: Reliable atom identification isn't just about training bigger autoencoders. The authors demonstrate a phase transition: high faithfulness and stability are only unlocked when the TSAE capacity strictly exceeds a critical threshold dictated by the scale of the training data.

Key Results

The authors evaluated their approach across Gemma2-2B, Gemma2-9B, and Llama3.1-8B:

  • Near-Perfect Metrics: The identified atoms achieved an average 99.90% faithfulness ($R^2$) and 99.85% stability ($q^*$) across all tested layers, statistically satisfying the criteria for ideal atoms.
  • High Monosemanticity: Using an automated LLM-as-a-judge (GPT-5.2) with manual verification, the extracted atoms demonstrated significantly higher monosemanticity scores compared to standard feature baselines from GemmaScope and LlamaScope.
Monosemanticity Scores Comparison

Limitations and Future Work

  • Scope of Models: While the mathematical foundations of Atom Theory rely on fundamental operations like matrix multiplication (suggesting applicability to architectures like diffusion models), the empirical validation in this paper is strictly limited to transformer-based LLMs.
  • Downstream Application: The paper focuses on establishing the theoretical framework, metrics, and validation rather than optimizing these atoms for specific downstream tasks like model editing, steering, or safety alignment.
  • Scalability: Future work is required to develop incremental, highly scalable methods to construct massive, model-wide atom repositories efficiently.

Summary updated

Abstract

The fundamental representational units (FRUs) of large language models (LLMs) remain undefined, limiting further understanding of their underlying mechanisms. In this paper, we introduce Atom Theory to systematically define, evaluate, and identify such FRUs, which we term atoms. Building on the atomic inner product (AIP), a non-Euclidean metric that captures the underlying geometry of LLM representations, we formally define atoms and propose two key criteria for ideal atoms: faithfulness ($R^2$) and stability ($q^*$). We further prove that atoms are identifiable under threshold-activated sparse autoencoders (TSAEs). Empirically, we uncover a pervasive representation shift in LLMs and demonstrate that the AIP corrects this shift to capture the underlying representational geometry. We find that two widely used units, neurons and features, fail to qualify as ideal atoms: neurons are faithful ($R^2=1$) but unstable ($q^*=0.5\%$), while features are more stable ($q^*=68.2\%$) but unfaithful ($R^2=48.8\%$). To find atoms of LLMs, leveraging atom identifiability under TSAEs, we show via large-scale experiments that reliable atom identification occurs only when the TSAE capacity matches the data scale. Guided by this insight, we identify FRUs with near-perfect faithfulness ($R^2=99.9\%$) and stability ($q^*=99.8\%$) across layers of Gemma2-2B, Gemma2-9B, and Llama3.1-8B, satisfying the criteria of ideal atoms statistically. Further analysis confirms that these atoms align with theoretical expectations and exhibit substantially higher monosemanticity. Overall, we propose and validate Atom Theory as a foundation for understanding the internal representations of LLMs. Code available at https://github.com/ChenhuiHu/towards_atoms.
BibTeX
@inproceedings{hu2026towards,
  author = {Chenhui Hu and Pengfei Cao and Yubo Chen and Kang Liu and Jun Zhao},
  title = {{Towards Atoms of Large Language Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=TaDbT9pZr7},
}
Figure