Introduction
Dimensionality reduction is a cornerstone of data science, allowing us to squash complex, high-dimensional datasets into 2D or 3D plots we can actually see. The goal is a "faithful" representation: if you see a cluster or a shape in the plot, it should represent a real relationship in the original data.
However, algorithms can sometimes create artefacts—patterns that are products of the math, not the data. This paper investigates a known but poorly understood phenomenon where Multidimensional Scaling (MDS) produces circular or "annular" structures even when the input data is completely "structureless" (isotropic). This is a major problem for researchers in fields like bioinformatics or psychology, who might spend months trying to explain a "ring" in their data that is actually just a mathematical ghost.
The core finding is that the popular SSTRESS measure (which uses squared distances) naturally forces structureless, high-dimensional data into a fake ring-shaped distribution, whereas the standard STRESS measure does not.
Methods and Datasets
The paper focuses on Metric Multidimensional Scaling (MDS), which tries to find a low-dimensional configuration of points such that the distances between them match the distances in the original high-dimensional space as closely as possible.
The authors compare two specific error functions:
- STRESS: Minimizes the difference between the actual distances ($d_{ij}$).
- SSTRESS: Minimizes the difference between the squared distances ($d_{ij}^2$).
SSTRESS is computationally popular because it’s easier to optimize (it’s the basis for the ALSCAL implementation in SPSS). The authors perform a rigorous theoretical analysis, calculating the derivative of the SSTRESS error and evaluating it under the assumption of high-dimensional isotropic data.
To isolate the algorithm's behavior, the authors used synthetic "structureless" datasets. They generated four datasets by randomly sampling 1,000 points from unit hypercubes with dimensions $p = 5, 10, 30,$ and $100$. Because these points are independent and identically distributed (i.i.d.), there is no inherent "ring" or "cluster" in the data; it is a uniform cloud.
Key Insights: The Annular Trap
- The Metric is the Message: The "aha moment" is that SSTRESS isn't just a faster version of STRESS; it changes the underlying geometry. By squaring the distances, the algorithm becomes much more sensitive to larger distances.
- The Curse of Dimensionality: In high-dimensional space, the distance between any two random points tends to concentrate around a single value.
- The Annular Trap: Because SSTRESS wants to satisfy all these nearly-equal high-dimensional distances simultaneously in 2D, the "mathematically easiest" solution is to place all points on a circle. This satisfies the requirement that most points are roughly the same distance from each other.
- Predictable Variance: The authors derived a formula showing that the variance of the map points ($\sigma_y^2$) is directly related to the data variance ($\sigma_x^2$) and the dimensionality ($p$): $$\sigma_y^2 \approx \frac{p}{q+1} \sigma_x^2$$ (where $q$ is the target dimension, e.g., 2).
Results and Visual Evidence
In plots of structureless data, STRESS produced a uniform cloud of points, accurately reflecting the lack of structure. However, SSTRESS produced a perfect, hollow ring—a pure artefact.
The authors compared their predicted map variances against observed results from their simulations. As the dimensionality $p$ increased, their prediction became incredibly accurate:
- At $p=5$, the prediction error was 16.4%.
- At $p=100$, the prediction error dropped to just 1.4%.
The paper mathematically proves that for high-dimensional isotropic data, the globally optimal solution for the SSTRESS measure is an annular (ring) shape.
Limitations and Broader Impact
The theoretical proof relies on the data being "structureless" (isotropic). While this is the perfect "null hypothesis" to prove the existence of artefacts, real-world data usually has some structure. The paper doesn't explore how this artefact interacts with or masks real, non-isotropic patterns.
Furthermore, while alternative transformations exist to fix this, they often assume the data follows a specific Gaussian distribution, which may not hold for real-world biomedical or sensor data.
Ultimately, the paper serves as a critical warning: if you are using SSTRESS (or software like SPSS that defaults to it) and you see a ring, be very skeptical. It might be time to switch back to standard STRESS or Sammon’s mapping to ensure you aren't just seeing ghosts in the data.
Summary updated