Rethinking Intrinsic Dimension Estimation in Neural Representations

Rickmer Schulte, David Rügamer

TL;DR

This paper reveals a critical flaw in how we measure the "intrinsic dimension" (ID) of neural network representations: common estimators don't actually track the true underlying dimension. Instead, they capture a phenomenon related to how representations spread out in latent space, which is more akin to entropy. This finding challenges interpretations of previous ID studies and suggests a new way to understand representation geometry.

What Problem Does This Paper Address?

Deep learning models are incredibly complex, operating in high-dimensional spaces. To understand them better, researchers often look at the "intrinsic dimension" (ID) of their internal representations. The idea is that even though these representations live in a high-dimensional space (like the pixels of an image), they might actually lie on or near a much lower-dimensional "manifold." This manifold hypothesis suggests that the "true" dimensionality of the data is much smaller than the ambient space.

However, previous studies have observed a consistent pattern: as data flows through a neural network, the estimated ID tends to increase in early layers and decrease in later layers. This has been interpreted as the emergence of abstractions or phase transitions. This paper questions whether these observed ID patterns truly reflect the underlying manifold's dimension or if there's a disconnect between theory and practice.

Methods

The authors tackle this problem through a combination of theoretical analysis and empirical investigation.

Theoretical Contributions: * They prove that for Lipschitz continuous neural network layers (which most standard layers are), the intrinsic dimension (both Hausdorff and pointwise) cannot increase as data passes through the network. This is a fundamental theoretical constraint. * They extend this to the "union of manifolds" hypothesis, showing the same non-increasing property holds. * They demonstrate that for Large Language Models (LLMs), due to the discrete nature of tokens, the intrinsic dimension of their embeddings and representations is theoretically zero.

Empirical Investigations: * They analyze layer-wise ID estimates from various vision models (CNNs, ViTs) and LLMs. * They investigate the underlying factors driving the observed ID patterns, focusing on: * NN distances: How distances between nearest neighbors change across layers. * Cosine Similarity: How similar representations are to each other. * Representation Length (L2 norm): How far representations are from the origin. * Entropy: A measure of how spread out the variance of representations is.

Datasets

The paper uses several standard datasets for their experiments:

  • ImageNet: For training and evaluating convolutional neural networks (CNNs) and vision transformers (ViTs).
  • Wikitext: For training and evaluating large language models (LLMs).

They also generate synthetic datasets (e.g., points sampled from a dm-hyperball) to control the true intrinsic dimension and study estimator behavior.

Key Insights and Ideas

  • ID Estimators are Misleading: The core insight is that common ID estimators (like TwoNN and MLE) do not track the true intrinsic dimension of neural representations. They are heavily biased, especially in high dimensions, and their layer-wise patterns are not indicative of the underlying manifold dimension.
  • The "ID Increase" is an Artifact: The observed increase in estimated ID in early layers is not due to the emergence of higher-dimensional abstractions. Instead, it's driven by the increasing separation of representations in latent space, which causes NN distances to grow.
  • Entropy is the Real Story: The paper strongly suggests that the observed layer-wise ID patterns are actually capturing changes in the entropy of the representation distribution. As representations spread out in latent space (increasing entropy), ID estimators tend to increase, and vice-versa. This provides a more theoretically grounded interpretation.
  • LLM Embeddings Have Zero ID: A surprising theoretical result is that LLM token embeddings, due to the discrete nature of tokens, have a strict topological intrinsic dimension of zero. This is fundamentally different from image representations.

Key Results

  • Theoretical Proofs: The paper provides mathematical proofs showing that intrinsic dimension cannot increase across Lipschitz layers in neural networks.
  • Empirical Evidence: Experiments across various LLMs, CNNs, and ViTs consistently show that estimated layer-wise IDs increase in early layers, contradicting the theoretical findings.
  • Bias in Estimators: Figures demonstrate a significant negative bias in common ID estimators, which grows with the true intrinsic dimension.
  • Correlation with Entropy: Layer-wise ID patterns strongly correlate with von Neumann entropy patterns across different model types, suggesting entropy is the underlying phenomenon being captured.
  • LLM Specifics: LLM embeddings and representations are shown to have a theoretical ID of zero.

Limitations and Future Work

Limitations: * The theoretical results on ID non-increase apply to Lipschitz layers. While most standard layers are Lipschitz, some operations (like hard quantization or certain types of self-attention) might violate this, though the authors discuss how to handle these in compact domains. * The paper focuses on pointwise and Hausdorff dimensions. Other notions of intrinsic dimension exist.

Future Work: * Develop Better Estimators: The authors suggest designing new ID estimators that are more reliable and adhere to theoretical constraints (e.g., non-increasing layer-wise IDs). * Formalize Entropy-ID Connection: Further investigation into the theoretical connection between ID and entropy is proposed as a promising direction. * Broader Implications: Re-evaluating conclusions drawn from previous ID estimation studies across various domains (vision, NLP) is recommended. * Understanding Representation Expansion: Further exploring why representations expand in latent space and how this expansion is structured (e.g., directionally) is suggested.

Summary updated

Abstract

The analysis of neural representation has become an integral part of research aiming to better understand the inner workings of neural networks. While there are many different approaches to investigate neural representations, an important line of research has focused on doing so through the lens of intrinsic dimensions (IDs). Although this perspective has provided valuable insights and stimulated substantial follow-up research, important limitations of this approach have remained largely unaddressed. In this paper, we highlight a crucial discrepancy between theory and practice of IDs in neural representations, theoretically and empirically showing that common ID estimators are, in fact, not tracking the true underlying ID of the representation. We contrast this negative result with an investigation of the underlying factors that may drive commonly reported ID-related results on neural representation in the literature. Building on these insights, we offer a new perspective on ID estimation in neural representations.
BibTeX
@inproceedings{schulte2026rethinking,
  author = {Rickmer Schulte and David Rügamer},
  title = {{Rethinking Intrinsic Dimension Estimation in Neural Representations}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2026},
  url = {https://openreview.net/forum?id=kH1gPRbYqh},
}

Loading related content...

Figure