Procrustes Analysis on the Manifold of SPSD Matrices for Data Sets Alignment

Almog Lahav, Ronen TalmonIEEE TSP 2023 · OpenReview

Abstract: In contemporary high-dimensional data analysis, intrinsically similar and related data sets are often significantly different due to various undesired factors that could arise from different acquisition equipment, calibration, environmental conditions, and many other sources of batch effects. Therefore, the task of aligning such data sets has become ubiquitous. In this work, we present a method for the alignment of different, but related, sets of Symmetric Positive Semidefinite (SPSD) matrices, which constitute a commonly-used family of features, e.g., covariance and correlation matrices, various kernels, and prototypical graph and network representations. Our method does not require any a-priori correspondence, and it is based on non-Euclidean Procrustes Analysis (PA) using a particular Riemannian geometry of SPSD matrices. While the derivation is focused on the manifold of SPSD matrices, we show that our alignment method can be applied directly in the original high-dimensional data space, when considering SPSD features that are sample covariance matrices. We demonstrate the advantage of our approach over competing methods in simulations and in an application to Brain-Computer Interface (BCI) with electroencephalographic (EEG) recordings.

Introduction: The Challenge of Batch Effects

In modern high-dimensional data analysis, we often encounter datasets that are intrinsically similar but appear statistically incompatible. These discrepancies, known as "batch effects," can stem from variations in acquisition equipment, environmental conditions, or subject-specific differences. For example, EEG recordings from two different individuals performing the same task may look vastly different due to physiological variations. Aligning these datasets is crucial for developing robust machine learning models that can generalize across different domains.

The Geometry of SPSD Matrices

Many data features, such as covariance matrices, kernels, and graph representations, naturally reside on curved manifolds rather than flat Euclidean spaces. Specifically, high-dimensional data often possesses a low-dimensional intrinsic structure, leading to features that are Symmetric Positive Semidefinite (SPSD) rather than strictly Symmetric Positive Definite (SPD).

While standard Procrustes Analysis (PA) assumes a Euclidean space, and some extensions exist for the SPD manifold, these methods often fail when matrices are rank-deficient (SPSD). Forcing SPSD data into SPD-based algorithms typically requires regularization "hacks" that can distort the underlying signal. This paper introduces a framework for non-Euclidean PA specifically tailored for the SPSD manifold with a fixed rank.

Non-Euclidean Procrustes Analysis

The proposed alignment method decomposes the process into three fundamental geometric operations on the manifold:

  1. Centering: A "Rigid Transport Map" moves the dataset's mean to a reference point (the manifold's origin) while preserving internal relationships.
  2. Scaling: The "dispersion" or spread of the data is matched between source and target sets along geodesic paths.
  3. Rotation: Using a small number of labeled landmarks, the method solves an optimization problem to align class-specific means.
The alignment process: (a) original datasets, (b) after centering, (c) after scaling, and (d) after rotation.

A significant theoretical finding is that these transformations, though derived via Riemannian geometry, can be applied directly to the original high-dimensional data matrices. This allows for the alignment of raw data before any downstream classification.

Experimental Evaluation

The method was validated using both synthetic and real-world datasets.

Toy Tori

To visualize the alignment and denoising capabilities, the authors used two 3D tori, where one is a scaled and rotated version of the other.

Representation of the toy tori datasets used for visualization.

BCI Application: MAMEM EEG Database

The framework was applied to a Brain-Computer Interface (BCI) task involving Steady-State Visually Evoked Potential (SSVEP) recordings from 11 subjects. The goal was to classify flickering frequencies based on 13 EEG channels.

Key Results: * Superior Accuracy: The SPSD-based alignment achieved a mean accuracy of 0.89, significantly outperforming standard Euclidean alignment (0.81) and traditional SPD-based methods (0.33), which struggled with rank deficiency. * Denoising Power: In simulations, the alignment improved the Signal-to-Noise Ratio (SNR) from 20.6 dB to 42 dB. * Visual Proof: t-SNE visualizations confirmed that post-alignment, clusters from different subjects overlapped almost perfectly, whereas they were disjoint beforehand.

Conclusion

By acknowledging the rank-deficient nature of real-world data, this SPSD-based approach captures true low-dimensional geometry. The method is highly label-efficient, as centering and scaling are unsupervised, requiring only a few labels for the final rotation. Future work aims to extend this to "dimensionality transcending" methods, allowing for the alignment of matrices with varying ranks or physical dimensions.

Summary updated

Abstract

In contemporary high-dimensional data analysis, intrinsically similar and related data sets are often significantly different due to various undesired factors that could arise from different acquisition equipment, calibration, environmental conditions, and many other sources of batch effects. Therefore, the task of aligning such data sets has become ubiquitous. In this work, we present a method for the alignment of different, but related, sets of Symmetric Positive Semidefinite (SPSD) matrices, which constitute a commonly-used family of features, e.g., covariance and correlation matrices, various kernels, and prototypical graph and network representations. Our method does not require any a-priori correspondence, and it is based on non-Euclidean Procrustes Analysis (PA) using a particular Riemannian geometry of SPSD matrices. While the derivation is focused on the manifold of SPSD matrices, we show that our alignment method can be applied directly in the original high-dimensional data space, when considering SPSD features that are sample covariance matrices. We demonstrate the advantage of our approach over competing methods in simulations and in an application to Brain-Computer Interface (BCI) with electroencephalographic (EEG) recordings.
BibTeX
@article{lahav2023procrustes,
  author = {Almog Lahav and Ronen Talmon},
  title = {{Procrustes Analysis on the Manifold of SPSD Matrices for Data Sets Alignment}},
  journal = {IEEE Trans. Signal Process.},
  year = {2023},
  url = {https://openreview.net/forum?id=XeUZPXUl9z},
}
Figure