Abstract
Detecting topological circularity in high-dimensional datasets is a critical task in fields ranging from biology to computer vision. This paper presents a novel approach to infer the existence of such structures in $\mathbb{R}^d$ by analyzing projections in $\mathbb{R}^2$ generated through a fast manifold learning map. This projection is a function of the high-dimensional dataset $\mathbb{X}$ and a bandwidth parameter $\sigma$. We propose a method to estimate the optimal bandwidth by minimizing an $L^2$ energy function and provide limit theorems to characterize the behavior of this function, ensuring a robust optimization framework.
Introduction
Manifold learning aims to recover low-dimensional structures embedded within high-dimensional data. Techniques like Laplacian Eigenmaps and Isomap are widely used but often suffer from two main drawbacks: high computational complexity and extreme sensitivity to the "bandwidth" parameter ($\sigma$). This parameter defines the scale of local neighborhoods; an incorrect choice can lead to either disconnected noise or a collapsed structure, obscuring the underlying geometry.
This research addresses the specific challenge of identifying topological circularity—where data points form a loop or periodic pattern. We propose a computationally efficient method to automatically determine the optimal bandwidth $\sigma$ by minimizing a specialized energy function, allowing for the reliable detection of circular manifolds in complex datasets.
The Fast SDD Manifold Learning Map
Traditional manifold learning often requires solving expensive eigenvalue problems. To overcome this, we utilize a framework based on Symmetric Diagonally Dominant (SDD) matrices. The projection is computed by solving a linear system, which can be achieved in nearly linear time—specifically $O(m \log^{1/2} n)$, where $m$ is the number of non-zero entries in the adjacency matrix. This efficiency makes the approach scalable to large-scale high-dimensional datasets.
Optimal Bandwidth via Energy Minimization
The core contribution of this work is the introduction of an $L^2$ energy function, $E(Z(\sigma))$, defined on the 2D projection $Z$.
- Energy Formulation: The function calculates the sum of squared distances between consecutive points in the projected space. Intuitively, this represents a squared version of the perimeter of the polygon formed by the projected data.
- Optimization Strategy: We show that the "sweet spot" for the bandwidth $\sigma$ occurs at the global minimum of this energy function. At this point, the projection most accurately represents a non-self-intersecting polygon, effectively "unfolding" the high-dimensional loop into a clear 2D circle.
- Theoretical Guarantees: We provide limit theorems proving that $E(Z(\sigma))$ and its derivatives are well-behaved as $\sigma$ approaches zero or infinity. These proofs ensure that a global minimum exists and can be found using standard optimization techniques like gradient descent combined with tunneling to avoid local minima.
Experimental Validation
The effectiveness of the proposed method was tested on both synthetic benchmarks and complex biological data.
Synthetic Data: The Toroidal Helix
We applied the algorithm to a toroidal helix—a 3D coil wrapped into a circular shape. By minimizing the $L^2$ energy function, the algorithm successfully identified an optimal $\sigma$ that projected this complex 3D structure into a clear, non-self-intersecting 2D loop, demonstrating the method's ability to recover the underlying topology.
Biological Insights: Malaria Resilience
In a real-world application, we analyzed a dataset from Stanford University involving the physiological traits (e.g., temperature, weight, red blood cell count) of mice infected with malaria. The goal was to distinguish between "resilient" mice (those that survived) and "non-resilient" mice.
- Resilient Mice: The optimal projection of their traits over time formed a distinct loop. This circularity indicates a return to a baseline state, reflecting the periodic nature of a successful recovery.
- Non-resilient Mice: Their data failed to form a closed loop, suggesting that their biological systems were unable to return to a "normal" state, which correlates with the clinical outcome of mortality.
Conclusion
By combining fast SDD-based manifold learning with a mathematically grounded bandwidth optimization strategy, we provide a robust tool for detecting periodicity and circularity in high-dimensional data. The $O(n)$ complexity of the energy calculation per iteration ensures that parameter tuning is fast, while the biological results underscore the practical utility of topological analysis in understanding complex systems. Future work will explore extending this framework to detect higher-dimensional topologies, such as spheres and tori.
Summary updated