Geometric Artifacts in Low Dimensional Projections

318 concepts242 papers1 proposal

When we visualize the inner workings of modern machine learning models—whether we are mapping the semantic space of large language model embeddings, tracing the optimization path of a neural network, or analyzing high-dimensional biological data—we almost always rely on dimensionality reduction (DR). Tools like t-SNE, UMAP, and PCA take vectors living in hundreds or thousands of dimensions and compress them into a 2D or 3D plot that a human can interpret.

However, this compression is not a neutral mathematical translation. Forcing high-dimensional geometry into a low-dimensional space introduces severe, non-intuitive distortions. As Hammer et al. (2013) note, these non-linear errors can easily mislead researchers into hallucinating patterns, clusters, and barriers that do not exist in the raw data. To trust our visual evaluations, we must understand the geometric mechanics that cause these artifacts.

The Crowding Problem and False Neighborhoods

The fundamental driver of projection distortion is a geometric mismatch: high-dimensional space has vastly more "room" than low-dimensional space.

Consider the volume of a sphere. In a 2D plane, the area within a radius $r$ scales as $r^2$. In a $D$-dimensional space, the volume scales as $r^D$. When $D$ is large (such as a 768-dimensional language model embedding space), there is an astronomical amount of orthogonal space available. This allows a large number of data points to be mutually equidistant and widely separated from one another.

When we attempt to map these points into a 2D visualization, we run into what Hinton and Roweis (2002) define as the crowding effect. Because 2D space lacks the volume to accommodate all these equidistant relationships, the projection algorithm is forced to collapse widely separated high-dimensional points into the same low-dimensional neighborhood.

This collapse creates false neighborhoods. As Gerling et al. (2025) demonstrate, points can appear tightly clustered on a 2D plot despite being highly distant in the original high-dimensional space. If a researcher relies solely on the visual proximity of these points, they may draw incorrect conclusions about model behavior or data similarity.

Spurious Clustering and the Illusion of Discreteness

To combat the crowding problem, non-linear dimensionality reduction algorithms like t-SNE and UMAP use optimization objectives that prioritize preserving local structures over global distances. They pull close neighbors together and push distinct groups apart. While this produces visually clean plots, it introduces a dangerous side effect: spurious clustering.

Huang et al. (2022) found that these algorithms frequently generate artificial separations and false clusters that do not correspond to any underlying structure in the data. This distortion is particularly problematic when visualizing continuous spaces. For instance, Islam et al. (2023) show that when applied to the continuous feature spaces of regression networks, standard DR techniques force the data into uninformative noise or distinct buckets, masking the continuous manifold the network has actually learned.

Furthermore, these algorithms distort density. Xia et al. (2021) establish that non-linear DR methods prioritize cluster separation at the expense of density preservation. A large, sparse cloud of points in high dimensions may look identical to a dense, compact cluster in 2D, making it impossible to evaluate the relative distribution of the underlying data from the visualization alone.

Misleading Landscapes and Hidden Paths

The distortion of high-dimensional geometry becomes even more acute when visualizing optimization surfaces, such as the loss landscapes of neural networks or the fitness landscapes of evolutionary biology.

We often visualize these spaces using 1D or 2D slices. However, Li et al. (2018) point out that low-dimensional slices inherently miss the geometric nuances of a million-dimensional parameter space. A 2D projection might show a steep "mountain" separating two local minima, implying an impassable optimization barrier. But in the true high-dimensional space, there are millions of alternative directions.

Jastrzebski (2019) demonstrates that these landscapes are often connected by low-loss "tunnels" that disappear entirely when projected down. Similarly, Louis et al. (2021) show that low-dimensional metaphors create a misleading perception of isolation between high-fitness regions in biological genotypes, whereas the high-dimensional space actually contains navigable, continuous bypasses. By flattening the space, we inadvertently invent obstacles and hide the trajectories that models actually take during training (Chatzimichailidis et al. (2019); Berard et al. (2020)).

Diagnosing and Mitigating Projection Artifacts

Because visual misinformation often arises from these unavoidable mathematical trade-offs rather than software bugs (Lo et al. (2022)), researchers have developed methods to quantify and expose projection artifacts.

Mathematical Verification Metrics

Instead of relying on subjective visual inspection, we can measure how well a map preserves the original space using quantitative metrics:

  • Topographic Product: Introduced by Bauer et al. (1991), this metric calculates the preservation of neighborhood orderings to determine mathematically whether a low-dimensional map is "folded" (indicating the target dimensionality is too low) or "squashed" (indicating it is too high).
  • Rank-Based Trustworthiness: Kaski and Nikkilä (2003) developed neighbor-analysis metrics that explicitly calculate the probability that points appearing close in a 2D plot are truly similar in the original high-dimensional space, allowing practitioners to flag false similarities.

Uncertainty-Aware Visual Analytics

Modern visualization tools now build these safeguards directly into the user interface. Cakmak et al. (2022) propose using dense visualizations of raw high-dimensional vectors alongside 2D projections to verify whether observed semantic groupings reflect true model behavior.

Taking this further, Gerling et al. (2025) introduce an interactive approach that explicitly links high-dimensional $k$-nearest neighbors within the low-dimensional plot. If two points appear adjacent in the 2D visualization but are not true neighbors in the high-dimensional space, the interface draws a visual overlay or highlights the discrepancy, immediately exposing the false neighborhood to the analyst.

Ultimately, the most robust defense against projection artifacts is a change in workflow: while low-dimensional plots are invaluable for initial, qualitative exploration, any definitive analytical conclusions—such as clustering, classification, or distance comparisons—should be computed directly within the native high-dimensional vector space (von Luxburg (2017)).

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 225 papers
  2. Version 1 · 2026-10-04 · 245 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_dimensionality_reduction_artifacts_11f93faf,
  title = {dimensionality reduction artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dimensionality-reduction-artifacts/rdr_11f93faf3a124579aabdcc17b0e9d5fe},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_dimensionality_reduction_artifacts_11f93faf_v2,
  title = {dimensionality reduction artifacts},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dimensionality-reduction-artifacts/rdr_11f93faf3a124579aabdcc17b0e9d5fe/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake