When we project high-dimensional data down to two or three dimensions for visualization, we are forced to make a geometric compromise. High-dimensional space is vast; there is simply more "room" for points to be mutually equidistant than there is in a low-dimensional plane. This fundamental constraint means that no projection can perfectly preserve all pairwise distances.
Historically, dimensionality reduction (DR) algorithms have chosen sides in this geometric tug-of-war. Linear methods like Principal Component Analysis (PCA) focus on preserving global variance, which often squashes distinct local clusters into an overlapping mass. Conversely, modern non-linear manifold learning techniques like t-SNE and UMAP prioritize local neighborhood preservation. While this local focus produces crisp, visually distinct clusters, it introduces a significant downside: it frequently distorts or completely tears the global topology of the data, rendering the relative distances between clusters meaningless.
Why Neighbor Embeddings Lose the Big Picture
To understand why popular visualization tools distort global structure, we have to look at how their loss functions handle distance. Consider Stochastic Neighbor Embedding (SNE), introduced by Hinton et al.. SNE converts high-dimensional Euclidean distances into conditional probabilities that represent similarities. A Gaussian distribution is centered over each point; if another point falls close by, its probability value is high, meaning the algorithm will exert a strong attractive force to keep them close in the low-dimensional space.
However, because the Gaussian distribution's tails decay exponentially, the probability of a distant point being a neighbor drops to effectively zero. As a result, if two clusters are far apart in the original space, the loss function assigns no penalty regardless of where those clusters land relative to one another in the final visualization.
As Huang et al. demonstrate, this neighborhood preservation bias causes continuous populations to fragment into artificial, disconnected clusters. For instance, in transcriptomic data, a continuous developmental pathway of cells might be shattered into discrete groups, obscuring the underlying biological trajectory.
Furthermore, as Xingyu Zhao notes, this hyper-focus on local proximity can even force outliers into the nearest visible clusters simply to satisfy local density constraints, creating a deceptively clean but unfaithful visualization.
High-Dimensional Space Purely Local Embedding (e.g., t-SNE)
[Cluster A] [Cluster A]
| (Distance = 10)
v [Cluster C]
[Cluster B] (Distance is arbitrary)
| (Distance = 10)
v [Cluster B]
[Cluster C]
(Global hierarchy is clear) (Global hierarchy is lost)
Designing Beyond the Trade-Off
For a long time, researchers treated the conflict between local cluster resolution and global topological fidelity as an unavoidable mathematical constraint. However, recent work reveals that this trade-off is actually a consequence of specific algorithmic design choices.
In an influential analysis, Cynthia Rudin demonstrated that by carefully structuring the optimization objective, it is entirely possible to maintain both fine-grained local details and broad global arrangements. This insight led to the development of algorithms like PaCMAP (Pairwise Controlled Manifold Approximation Projection), detailed by Wang et al..
Instead of relying on a single probability distribution across all distances, PaCMAP explicitly manipulates three distinct types of pairs during its optimization process:
- Neighbor pairs: Chosen from the immediate local neighborhood to preserve local clusters.
- Mid-range pairs: Sampled to maintain the broader, mid-level structure and prevent clusters from drifting arbitrarily.
- Further pairs: Drastically separated pairs that enforce a global repulsive force, ensuring that distant clusters remain distant.
By dynamically altering the weights of these forces across different stages of optimization, the algorithm prevents the formation of uninterpretable "hairballs" while simultaneously preserving the macro-scale geometry of the dataset. As Haiyang Huang shows, enhancing the separation of local clusters in such a parametric framework does not inherently degrade the global arrangement of data, proving that local and global preservation are not mutually exclusive.
Similarly, Zhirong Yang et al. proposed Stochastic Cluster Embedding (SCE), which adaptively balances attractive and repulsive forces. SCE intentionally relaxes strict local neighborhood preservation at specific scales to enhance the visibility of global cluster boundaries, preventing distinct macro-structures from collapsing into a single continuous mass.
Alternative Topological and Geometric Paradigms
Beyond tweaking force-directed graph layouts, researchers have explored entirely different mathematical frameworks to bridge the local-global divide.
Differentiable Topology
One prominent direction involves using algebraic topology to constrain the embedding. Moor et al. introduced Topological Autoencoders (TopoAE), which integrate persistent homology into a deep learning framework. Instead of optimizing purely for reconstruction or local neighbor accuracy, TopoAE identifies the multi-scale topological features (such as connected components, loops, and voids) in both the input space and the latent space. By minimizing the difference between these topological signatures via a differentiable loss term, the model forces the low-dimensional representation to maintain the global "shape" and connectivity of the original data manifold.
On a linear front, Oleg Kachan developed a persistent homology-based projection pursuit algorithm that optimizes over the Stiefel manifold, allowing linear projections to capture global structures like holes or cycles that traditional methods like PCA would completely flatten.
Non-Euclidean Target Spaces
Another approach changes the geometry of the target space itself. Because high-dimensional hierarchical data expands exponentially, forcing it into a flat 2D Euclidean plane inevitably causes distortion. To combat this, Fischer et al. developed MERCAT, a method that abandons traditional distance preservation in favor of preserving angles between data points on a low-dimensional sphere. This spherical embedding naturally captures both fine-grained local clusters and broad global orientations, mitigating the "tearing" common to flat embeddings.
Ultimately, the evolution of dimensionality reduction has shifted from a binary choice between local and global fidelity toward unified frameworks. Whether through multi-scale force optimization, topological regularization, or non-Euclidean geometries, modern methods aim to ensure that when we look at a 2D plot, we can trust both the internal structure of a cluster and its position relative to the rest of the data universe.