RATS: Unsupervised manifold learning using low-distortion alignment of tangent spaces

Dhruv Kohli, Johannes S Nieuwenhuis, Alexander Cloninger, Gal Mishne, Devika NarainbioRxiv 2024 · DOI

Abstract

With the ubiquity of high-dimensional datasets in various biological fields, identifying low-dimensional topological manifolds within such datasets may reveal principles connecting latent variables to measurable instances in the world. The reliable discovery of such manifold structure in high-dimensional datasets can prove challenging, however, largely due to the introduction of distortion by leading manifold learning methods. The problem is further exacerbated by the lack of consensus on how to evaluate the quality of the recovered manifolds. Here, we present a novel measure of distortion to evaluate low-dimensional representations obtained using different techniques. We additionally develop a novel bottom-up manifold learning technique called Riemannian Alignment of Tangent Spaces (RATS) that aims to recover low-distortion embeddings of data, including the ability to embed closed manifolds into their intrinsic dimension using a unique tearing process. Compared to previous methods, we show that RATS provides low-distortion embeddings that excel in the visualization and deciphering of latent variables across a range of idealized, biological, and surrogate datasets that mimic real-world data.

Introduction: The Challenge of Manifold Distortion

The "Manifold Hypothesis" suggests that complex, high-dimensional data—such as the firing patterns of thousands of neurons—often resides on a much simpler, low-dimensional surface (a manifold). Dimensionality reduction aims to "unfold" these surfaces into interpretable 2D or 3D representations. However, existing methods like t-SNE and UMAP frequently introduce significant geometric distortion, warping distances and shapes in ways that can lead to incorrect scientific interpretations. For example, distances on a manifold might represent physical time or space; warping these distances creates "algorithmic artifacts" that obscure the underlying biology.

To address this, researchers developed RATS (Riemannian Alignment of Tangent Spaces), a bottom-up manifold learning algorithm designed to minimize distortion and preserve the physical meaning of latent variables.

Methods: A New Metric and a Rigid Alignment Strategy

The RATS framework introduces two major innovations: a robust evaluation metric and a novel embedding algorithm.

1. Global Distortion Metric ($\mathbb{G}_k$)

The authors propose $\mathbb{G}_k$ to quantify how much an embedding warps the "geodesics" (shortest paths) between points. It calculates the ratio between the maximum expansion and maximum contraction of these paths. A value of 1 indicates perfect geometric preservation, providing a standardized yardstick for manifold quality.

2. The RATS Algorithm

RATS operates through a three-step process:

  • Local Views: It generates low-distortion local snapshots of the data using Kernel PCA.
  • Rigid Alignment: These snapshots are stitched together using Riemannian Gradient Descent. By enforcing rigid transformations (rotations and translations), RATS avoids the "squishing" artifacts common in affine-based methods like LTSA.
  • Manifold Tearing: RATS can detect closed manifolds (e.g., spheres or tori) and iteratively "tear" them to allow flat embeddings into their true intrinsic dimension. It provides color-coded "gluing instructions" to show how the edges should reconnect.
Overview of the RATS algorithm, including the global distortion metric and the process of alignment and tearing.

Results: Benchmarking Low-Distortion Embeddings

The authors validated RATS against standard techniques (t-SNE, UMAP, ISOMAP, LDLE) across idealized shapes, biological neural recordings, and deep learning surrogate datasets.

Superior Geometric Preservation

In tests ranging from simple Swiss rolls to complex non-orientable Klein bottles, RATS consistently produced lower global distortion than competing methods. While methods like UMAP often fragment or warp closed manifolds, RATS successfully recovers the underlying structure with minimal artifacts.

Performance comparison of RATS against standard manifold learning methods across diverse topological shapes and noise conditions.

Biological Insights and Decoding Accuracy

In biological applications, such as decoding mouse head-direction cell activity, RATS achieved the lowest Mean Squared Error (MSE) compared to other unsupervised methods. By preserving the geometry of the neural manifold, RATS allows for more accurate mapping of latent variables to physical behaviors. Furthermore, it successfully identified the toroidal structure of grid cell activity, aligning with theoretical topological predictions.

Discussion and Future Directions

RATS represents a significant step toward "artifact-free" dimensionality reduction. By prioritizing geometric rigidity and introducing a principled way to handle closed manifolds through tearing, it provides a more reliable lens for exploring high-dimensional biological data.

Current limitations include sensitivity to "shortcuts" in noisy data—a common issue for geodesic-based methods—and the computational cost of calculating all-pairs geodesics. Future work will focus on scaling the algorithm and applying it to more abstract domains, such as language models or complex social dynamics, where preserving latent geometry is critical for understanding system behavior.

Summary updated

Abstract

With the ubiquity of high-dimensional datasets in various biological fields, identifying low-dimensional topological manifolds within such datasets may reveal principles connecting latent variables to measurable instances in the world. The reliable discovery of such manifold structure in high-dimensional datasets can prove challenging, however, largely due to the introduction of distortion by leading manifold learning methods. The problem is further exacerbated by the lack of consensus on how to evaluate the quality of the recovered manifolds. Here, we present a novel measure of distortion to evaluate low-dimensional representations obtained using different techniques. We additionally develop a novel bottom-up manifold learning technique called Riemannian Alignment of Tangent Spaces (RATS) that aims to recover low-distortion embeddings of data, including the ability to embed closed manifolds into their intrinsic dimension using a unique tearing process. Compared to previous methods, we show that RATS provides low-distortion embeddings that excel in the visualization and deciphering of latent variables across a range of idealized, biological, and surrogate datasets that mimic real-world data.
BibTeX
@misc{kohli2024rats,
  author = {Dhruv Kohli and Johannes S Nieuwenhuis and Alexander Cloninger and Gal Mishne and Devika Narain and Devika Narain},
  title = {{RATS: Unsupervised manifold learning using low-distortion alignment of tangent spaces}},
  year = {2024},
  doi = {10.1101/2024.10.31.621292},
  howpublished = {bioRxiv},
}
Figure