GyroAtt: A Gyro Attention Framework for Matrix Manifolds
Lists
Only you see this
Abstract
Deep neural networks operating on non-Euclidean geometries, such as Riemannian manifolds, have recently demonstrated impressive performance across various machine-learning applications. Motivated by the success of the attention mechanism, several works have extended it to different geometries. However, existing Riemannian attention methods are mostly designed in an \textit{ad hoc} manner, \textit{i.e.}, tailored to a selected few geometries. Recent studies, on the other hand, show that several matrix manifolds, such as Symmetric Positive Definite (SPD), Symmetric Positive Semi-Definite (SPSD), and Grassmannian manifolds, admit gyro structures, offering a principled way to build Riemannian networks. Inspired by this, we propose a Gyro Attention (GyroAtt) framework over general gyro spaces, applicable to various matrix manifolds. Empirically, we manifest our framework on three gyro structures in the SPD manifold, three in the SPSD manifold, and one in the Grassmannian manifold. Extensive experiments on four electroencephalography (EEG) datasets demonstrate the effectiveness of the proposed framework.
BibTeX
@misc{wang2025gyroatt,
author = {Rui Wang and Chen Hu and Xiaojun Wu and Xiaoning Song and Nicu Sebe and Ziheng Chen},
title = {{GyroAtt: A Gyro Attention Framework for Matrix Manifolds}},
year = {2025},
url = {https://openreview.net/forum?id=YcaFqY8LWD},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}