In modern machine learning and biology, "seeing is believing." When researchers want to understand how a Large Language Model organizes concepts or how a single cell differentiates, they turn to dimensionality reduction (DR) tools like t-SNE and UMAP. These algorithms take high-dimensional vectors—often with thousands of dimensions—and squash them into a 2D map. When we see neat, separated clusters on these maps, we assume we have discovered the "natural" structure of the data.
However, a growing body of evidence suggests these visualizations are often "hallucination machines." As Noah Bergam and Nakul Verma prove in t-SNE Exaggerates Clusters, Provably, these tools can make perfectly uniform data look like distinct clusters, creating scientific "discoveries" out of mathematical noise.
The Mechanism of Spurious Clustering
The primary goal of neighbor-embedding methods like t-SNE and UMAP is to preserve local neighborhoods: if two points are close in high-dimensional space, they should be close in the 2D projection. To achieve this, these algorithms use a balance of attraction (pulling neighbors together) and repulsion (pushing non-neighbors apart).
The "hallucination" occurs when the repulsion force is too aggressive or the attraction is too localized. Sebastian Damrich and Fred A Hamprecht found that UMAP’s success often stems from a specific "negative sampling" balance that effectively binarizes the data graph. This creates "crisp" clusters and string-like structures that may not exist in the original space UMAP does not reproduce high-dimensional similarities due to negative sampling.
When practitioners see these crisp gaps, they interpret them as evidence of distinct categories. In reality, the algorithm may have simply "over-contracted" the data, artificially pushing overlapping groups apart Assessing and improving reliability of neighbor embedding methods: a map-continuity perspective.
The Dimensionality Gap and Topological Loss
The fundamental problem is the dimensionality gap. A two-dimensional plane simply does not have the "room" to represent the complex connectivity of a million-dimensional space.
This is most apparent when visualizing the loss landscapes of neural networks. Hao Li and colleagues demonstrated that while 2D slices can show us if a loss surface is "chaotic" or "smooth," they are physically incapable of proving global properties like convexity Visualizing the Loss Landscape of Neural Nets. A 2D projection might show a high-loss "ridge" between two solutions, but in the full high-dimensional space, those solutions might be connected by a hidden "tunnel" or path that the projection completely obscured An Empirical Analysis of Deep Network Loss Surfaces.
In biological data, this gap leads to the destruction of global structure. While t-SNE and UMAP excel at showing local cell types, they often distort the "global" distances between those types. Haiyang Huang and Cynthia Rudin note that this sensitivity to parameters can lead researchers to interpret algorithmic quirks as meaningful biological transitions Towards a comprehensive evaluation of dimension reduction methods for transcriptomic data visualization.
From Qualitative Maps to Quantitative Proof
Because of these distortions, researchers are moving away from using 2D embeddings as "proof" and toward more robust diagnostic tools. Several strategies have emerged to bridge the gap:
- Local Neighborhood Analysis: Instead of projecting the data, methods like the $k^*$ distribution analyze the latent space directly. By measuring how many "neighbors" of the same class surround a point in high-D, Shashank Kotyan and Danilo Vasconcellos Vargas can determine if a cluster is truly cohesive or just a visual illusion k* Distribution: Evaluating the Latent Space of Deep Neural Networks using Local Neighborhood Analysis.
- Uncertainty Quantification: New frameworks like DE-VAE add "confidence gauges" to projections. By using differential entropy, these models can visualize the inherent uncertainty of a mapping, showing users which parts of a 2D map are reliable and which are highly distorted DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy.
- Alternative Geometries: Some data structures, like the branching hierarchies of stem cells, are mathematically impossible to embed in Euclidean (flat) 2D space without massive distortion. Anna Klimovskaia and Leon Bottou propose using Poincaré maps—which use hyperbolic geometry—to capture these hierarchies with far greater fidelity Poincaré Maps for Analyzing Complex Hierarchies in Single-Cell Data.
The consensus across these 178 papers is clear: low-dimensional visualizations are powerful for hypothesis generation, but they are dangerous when used as quantitative evidence. As Sebastian Damrich and colleagues argue, we should treat these maps as a "spectrum" of possible views rather than a single, definitive truth Visualizing single-cell data with the neighbor embedding spectrum.