Spectral Graph Transformer Networks for Brain Surface Parcellation

Ran He, Karthik Gopinath, Christian Desrosiers, Herve Lombaert

Abstract

The analysis of the brain surface modeled as a graph mesh is a challenging task. Conventional deep learning approaches often rely on data lying in the Euclidean space. As an extension to irregular graphs, convolution operations are defined in the Fourier or spectral domain. This spectral domain is obtained by decomposing the graph Laplacian, which captures relevant shape information. However, the spectral decomposition across different brain graphs causes inconsistencies between the eigenvectors of individual spectral domains, causing the graph learning algorithm to fail. Current spectral graph convolution methods handle this variance by separately aligning the eigenvectors to a reference brain in a slow iterative step. This paper presents a novel approach for learning the transformation matrix required for aligning brain meshes using a direct data-driven approach. Our alignment and graph processing method provides a fast analysis of brain surfaces. The novel Spectral Graph Transformer (SGT) network proposed in this paper uses very few randomly sub-sampled nodes in the spectral domain to learn the alignment matrix for multiple brain surfaces. We validate the use of this SGT network along with a graph convolution network to perform cortical parcellation. Our method on 101 manually-labeled brain surfaces shows improved parcellation performance over a no-alignment strategy, gaining a significant speed (1400 fold) over traditional iterative alignment approaches.

Introduction

The human brain's cortical surface is a complex, folded 3D manifold. Modeling this surface as an irregular graph mesh allows researchers to leverage graph neural networks (GNNs) for tasks like parcellation—segmenting the brain into functional or anatomical regions. To handle the non-Euclidean nature of these meshes, spectral graph convolutions are often employed. These operations work in the spectral domain, derived from the eigenvectors of the graph Laplacian.

However, a significant hurdle exists: the spectral embeddings of different brains are not naturally aligned. Geometric variations lead to inconsistencies such as sign flips, eigenvector reordering, and mixing at higher frequencies. Without alignment, a GNN cannot generalize across different subjects. Traditional solutions rely on iterative mathematical alignment (e.g., Procrustes analysis) to a reference template. While effective, this preprocessing step is computationally expensive, creating a bottleneck in large-scale neuroimaging pipelines.

Methodology: The Spectral Graph Transformer (SGT)

To address the limitations of iterative alignment, we propose the Spectral Graph Transformer (SGT) network. This architecture learns the necessary transformation matrix directly from the spectral data in a single forward pass.

Overview of the SGT architecture and parcellation pipeline.

Architecture Overview

The pipeline begins by mapping brain graphs to a low-dimensional manifold using the first three components of the normalized graph Laplacian. The SGT module then takes a random sub-sample of $N$ nodes from this spectral domain. By processing these points through point-wise linear transformations and Multi-Layer Perceptrons (MLPs), the SGT predicts a 3x3 orthogonal transformation matrix $T(\theta_t)$.

A key finding of our work is that the SGT does not require the full mesh (often >150,000 vertices) to determine alignment. Randomly sampling just 1,000 nodes provides sufficient geometric context to capture the global shape of the embedding while drastically reducing computational overhead.

Graph Convolution and Joint Training

The aligned spectral coordinates are passed into a 4-layer Graph Convolutional Network (GCN) utilizing Gaussian kernels. This network performs the final node-wise classification into 32 anatomical parcels. The SGT and GCN are trained end-to-end. The loss function is a weighted combination of a spectral alignment loss (Mean Squared Error against traditional methods plus an orthogonality constraint) and a parcellation loss (Cross-entropy and Dice loss).

Results and Discussion

We evaluated our approach on the Mindboggle dataset, consisting of 101 manually-labeled cortical meshes. Our results demonstrate that data-driven alignment is not only viable but exceptionally efficient.

Qualitative and quantitative comparison of parcellation results across different alignment strategies.

Performance and Speed

As shown in the comparison above, the SGT achieves a Dice overlap of 83.2%, which is a substantial improvement over the 78.8% achieved with no alignment. While it slightly trails the traditional iterative method (84.4%), the trade-off in speed is transformative. The SGT performs alignment in just 10.7 milliseconds, compared to 15 seconds for the traditional approach—a 1400-fold speedup.

Conclusion

The Spectral Graph Transformer provides a fast, differentiable alternative to traditional spectral alignment. By learning to align brain surfaces directly from data, we enable near-instantaneous preprocessing for graph-based brain analysis. This framework is highly scalable and can be extended beyond parcellation to other surface-based tasks, such as disease classification or biomarker discovery, where speed and geometric accuracy are paramount.

Summary updated

Abstract

The analysis of the brain surface modeled as a graph mesh is a challenging task. Conventional deep learning approaches often rely on data lying in the Euclidean space. As an extension to irregular graphs, convolution operations are defined in the Fourier or spectral domain. This spectral domain is obtained by decomposing the graph Laplacian, which captures relevant shape information. However, the spectral decomposition across different brain graphs causes inconsistencies between the eigenvectors of individual spectral domains, causing the graph learning algorithm to fail. Current spectral graph convolution methods handle this variance by separately aligning the eigenvectors to a reference brain in a slow iterative step. This paper presents a novel approach for learning the transformation matrix required for aligning brain meshes using a direct data-driven approach. Our alignment and graph processing method provides a fast analysis of brain surfaces. The novel Spectral Graph Transformer (SGT) network proposed in this paper uses very few randomly sub-sampled nodes in the spectral domain to learn the alignment matrix for multiple brain surfaces. We validate the use of this SGT network along with a graph convolution network to perform cortical parcellation. Our method on 101 manually-labeled brain surfaces shows improved parcellation performance over a no-alignment strategy, gaining a significant speed (1400 fold) over traditional iterative alignment approaches.
BibTeX
@inproceedings{he2020spectral,
  author = {Ran He and Karthik Gopinath and Christian Desrosiers and Herve Lombaert},
  title = {{Spectral Graph Transformer Networks for Brain Surface Parcellation}},
  booktitle = {ISBI},
  year = {2020},
  doi = {10.1109/isbi45749.2020.9098737},
}
Figure