Comparing datasets that lack point-to-point alignment—whether due to different sizes, dimensions, or feature spaces—is a fundamental challenge in machine learning. In many real-world scenarios, we need to measure the similarity between two collections of data. For example, we might want to compare medical datasets from two different hospitals, track how the "shape" of data changes as it passes through different layers of a neural network, or determine if a new task in a few-shot learning setting is similar enough to training tasks to succeed.
The "gold standard" for comparing datasets often involves finding an alignment (matching a point in dataset 1 to a point in dataset 2). However, finding this alignment is an NP-hard problem. Existing "alignment-free" methods often ignore the underlying geometric structure of the data or are computationally prohibitive for large-scale applications.
To address this, researchers have introduced the Log-Euclidean Signature (LES) distance, a geometrically-grounded metric that compares the intrinsic manifold structure of datasets using a computationally efficient lower bound of the Log-Euclidean Riemannian metric.
Methodology: Manifolds and Spectral Signatures
The authors leverage manifold learning and Riemannian geometry to build an intrinsic, alignment-free, and efficient distance metric:
- Diffusion Operators: Each dataset is represented as a Symmetric Positive-Definite (SPD) matrix called a diffusion operator. This matrix captures how "information" diffuses across the data manifold, effectively approximating the manifold's underlying geometry.
- The Log-Euclidean (LE) Metric: SPD matrices live on a curved Riemannian manifold rather than a flat Euclidean space. The LE metric is a standard way to measure distances on this manifold, but it typically requires the matrices to be the same size and aligned.
- The LES Distance: The core innovation is a mathematical lower bound of the LE metric. The authors prove that the distance between two SPD matrices is lower-bounded by the Euclidean distance between the logarithms of their ordered eigenvalues.
- Spectral Signatures: By using only the top $K$ eigenvalues, they create a "signature" for each dataset. This allows them to compare datasets of different sizes (rows) and different feature spaces (columns) simply by comparing these $K$-dimensional vectors.
- Scalability: To handle massive datasets, they use Nyström-based spectral approximation, allowing them to estimate these signatures without ever materializing the full $N \times N$ diffusion matrix.
Key Insights and Results
The authors evaluated LES across several diverse domains, including 3D/4D point clouds, massive single-cell RNA-sequencing (scRNA-seq) datasets, few-shot learning benchmarks, and neural network embeddings.
Geometric Sensitivity
Unlike simple pointwise comparisons, LES respects the SPD structure of the diffusion kernel. This makes it significantly more sensitive to fine-grained structural differences than previous methods like Intrinsic Multi-scale Distance (IMD).
As demonstrated on toy geometric shapes (tori with varying radii), LES provides a clear, monotonic distance response to geometric scaling, whereas other metrics like IMD or Gromov-Wasserstein (GW) can struggle to capture these specific structural shifts consistently.
Efficiency and Robustness
LES is orders of magnitude faster than Gromov-Wasserstein, making it practical for comparing dozens of large datasets simultaneously. Furthermore, LES proved much more stable across different sample sizes than competitors like GW or Geometry Score (GS).
Real-World Applications
- Biological Accuracy: In a scRNA-seq experiment of ~250,000 mouse embryonic cells, LES recovered the developmental trajectory of cells with a higher correlation to the actual time axis (0.74) compared to IMD (0.28).
- Predicting Meta-Learning Failure: In meta-learning, LES can quantify how "far" a test task is from the training distribution. On the challenging FC100 benchmark, LES achieved a near-perfect negative correlation (-0.94) between task distance and few-shot accuracy, vastly outperforming IMD (-0.18) and Wasserstein distance (0.73, positive correlation). This provides a way to anticipate model failure without needing labels.
- Alignment-Free Layer Matching: LES successfully identified corresponding layers in two different neural networks even when they were trained with different random seeds and fed different input samples.
Limitations and Future Work
While powerful, LES has a few limitations. A known theoretical limitation in spectral geometry is that two different shapes can occasionally have the same eigenvalues (isospectral manifolds). While rare in practice, such datasets would be indistinguishable to LES. Additionally, the method requires choosing a kernel scale ($\sigma$) and a regularization parameter ($\gamma$), which may require domain-specific tuning.
Looking forward, LES opens up exciting possibilities for diagnosing distribution shifts in production ML models and guiding the design of more robust neural network architectures by analyzing their internal geometric signatures.
Summary updated