Abstract
In many dimensionality reduction tasks, we wish to identify the constituent components that explain our observations. For manifold learning, this can be formalized as factoring a Riemannian product manifold. Recovering this factorization, however, may suffer from certain difficulties in practice, especially when data is sparse or noisy, or when one factor is distorted by the other. To address these limitations, we propose identifying non-redundant coordinates on the product manifold before applying product manifold learning to identify which coordinates correspond to different factor manifolds. We demonstrate our approach on both synthetic and real-world data.
Introduction
Manifold learning aims to uncover the low-dimensional structure of high-dimensional data. Often, these structures are product manifolds—combinations of simpler components, such as a torus representing two independent rotations. While Product Manifold Learning (PML) is the current state-of-the-art for factorizing these structures, it faces significant practical hurdles. Specifically, spectral methods often produce redundant harmonic eigenvectors, and real-world data frequently exhibits warped geometries or extreme aspect ratios that cause standard PML to fail.
Independent Coordinate Selection (IC-PML)
To overcome these challenges, we introduce Independent Coordinate Product Manifold Learning (IC-PML). The core innovation is a pre-processing step that filters the embedding coordinates. Instead of relying on the first $N$ eigenvectors of the graph Laplacian, IC-PML employs a greedy search to select a subset of eigenvectors that maximize "rank quality." This is quantified by the normalized projected volume, ensuring that selected coordinates are geometrically independent and effectively span the manifold's tangent space.
By restricting the subsequent PML "triple identification" process—where factors are found via relationships like $\phi_k \approx \phi_i \phi_j$—to this independent set, we prevent the algorithm from misidentifying redundant harmonics as distinct factors. This refinement ensures that the algorithm captures the true underlying degrees of freedom rather than mathematical artifacts.
Experimental Results
We evaluated IC-PML against standard PML on synthetic and real-world datasets. A key test involved rectangular strips with varying aspect ratios. As the aspect ratio increases, the dominant eigenvectors primarily capture the long dimension, causing the shorter dimension to "collapse" in standard PML embeddings.
As shown in the figures above, for a $1 \times 7$ rectangle, standard PML collapses the embedding into a one-dimensional curve. In contrast, IC-PML's greedy coordinate selection identifies the high-frequency eigenvectors necessary to preserve the two-dimensional structure. This robustness is critical for accurately modeling datasets where factors have disparate scales.
Similar improvements were observed in more complex scenarios:
- Warped Cylinders: IC-PML correctly separated height from rotation, whereas PML produced topologically trivial results.
- Rotating Puppets: In a video of two rotating figures, IC-PML disentangled the rotations into two distinct circles, while PML failed to capture the product structure.
- Plowed Fields: The method successfully identified factor structures in warped real-world image data by loosening eigenvalue criteria.
Conclusion and Future Work
IC-PML significantly enhances the robustness of manifold factorization by prioritizing geometric independence over simple eigenvalue ordering. This approach is particularly effective for datasets with large aspect ratios or warped geometries. Future work will focus on optimizing the computational efficiency of the greedy search and exploring applications in decoding multi-factor biological or physical signals from complex data.
Summary updated