Abstract
Manifold alignment has been found to be useful in many areas of machine learning and data mining. In this paper we introduce a novel manifold alignment approach, which differs from "semi-supervised alignment" and "Procrustes alignment" in that it does not require predetermining correspondences. Our approach learns a projection that maps data instances (from two different spaces) to a lower dimensional space simultaneously matching the local geometry and preserving the neighborhood relationship within each set. This approach also builds connections between spaces defined by different features and makes direct knowledge transfer possible. The performance of our algorithm is demonstrated and validated in a series of carefully designed experiments in information retrieval and bioinformatics.
Introduction
In machine learning, high-dimensional data often resides on a lower-dimensional "manifold." Manifold alignment seeks to find common structures between two such datasets. Traditional methods, such as Procrustes or semi-supervised alignment, require pre-existing correspondences—known "anchor points" that link specific instances across datasets. However, in many real-world scenarios, such as transferring control policies between different robots or aligning biological data from different species, these correspondences are unknown. This paper introduces a novel manifold alignment technique that bridges disparate datasets without requiring any pre-existing point-to-point correspondences.
Methods
The proposed algorithm shifts the focus from matching individual points to matching local geometries. It follows a four-step process:
- Local Geometry Matching: For every point, the algorithm analyzes its $k$-nearest neighbors to construct an "elementary contact pattern." These patterns are compared across datasets using a rotation-and-scale-invariant distance metric. Similar "neighborhood vibes" indicate likely candidates for alignment.
- Joint Structure Construction: These local similarities are used to build a "joint" graph connecting the two manifolds.
- Optimal Projection: The algorithm solves a generalized eigenvalue problem to find linear mapping functions ($\alpha$ and $\beta$) that project both datasets into a shared lower-dimensional space $Z$. This step simultaneously preserves internal relationships and pulls similar local geometries together.
- Correspondence Discovery: Once in the shared space $Z$, finding matches becomes a simple nearest-neighbor search.
Unlike previous methods that only compute embeddings for known points, this approach learns mapping functions, allowing new data points to be projected into the aligned space without re-running the algorithm.
Experimental Results
The approach was validated in two distinct domains: bioinformatics (protein structures) and information retrieval (NIPS papers).
Bioinformatics: Protein Structures
Using ensembles of 3D protein models generated from NMR data, the algorithm successfully aligned protein manifolds across 3D, 2D, and 1D spaces, effectively "unfolding" them into a shared coordinate system. This demonstrates the algorithm's ability to preserve structural integrity across transformations.
Information Retrieval: NIPS Papers
The algorithm was tested on 1,740 NIPS papers represented in two different "topic spaces" (Latent Semantic Indexing and a Diffusion Model). The goal was to recognize the same paper across these different mathematical representations. The algorithm achieved a 65% Top-1 accuracy and an 80% Top-4 accuracy.
As shown in the results, the unsupervised approach performed remarkably well, nearly matching the performance of a semi-supervised version that utilized 15% prior correspondences, and significantly outperforming baselines that only consider local geometry.
Conclusion
This paper demonstrates that complex data can be aligned without a "Rosetta Stone" of correspondences by leveraging local geometric similarities. By seeking linear mapping functions rather than just coordinates, the model becomes a reusable tool for translating between disparate feature spaces. Future work will explore non-linear mapping functions to handle more complex manifold distortions.
Summary updated