Spectral Graph Transformer Networks for Brain Surface Parcellation

Ran He, Karthik Gopinath, Christian Desrosiers, Herve Lombaert

TL;DR

Analyzing brain surfaces using graph neural networks is challenging because the spectral embeddings (eigenvectors) of different brains do not naturally align. This paper introduces a Spectral Graph Transformer (SGT) network that learns to align these spectral domains directly from data, replacing a slow iterative preprocessing step. When applied to brain surface parcellation, the SGT achieves a massive 1400x speedup over traditional alignment methods while maintaining highly accurate segmentation.

What Problem Does This Paper Address?

The surface of the human brain is a highly complex, folded 3D structure. To analyze it using deep learning, researchers often model the brain as an irregular graph mesh. Because standard Convolutional Neural Networks (CNNs) require flat, Euclidean data (like images), researchers use spectral graph convolutions, which operate in the Fourier/spectral domain by decomposing the graph Laplacian.

However, there is a major catch: the spectral eigenbases of different brains are fundamentally incompatible. Due to variations in brain geometry, the eigenvectors suffer from sign flips, different ordering, and mixing at higher frequencies. If you feed unaligned spectral data into a neural network, it fails to learn effectively. Currently, the standard fix is to use an iterative mathematical algorithm (like Procrustes) to align each brain's eigenvectors to a reference template. This step is a severe computational bottleneck, taking seconds per brain and slowing down the entire training pipeline.

Methods

The authors propose an end-to-end deep learning architecture that learns the alignment matrix directly from the data, bypassing the slow iterative process.

  1. Spectral Embedding: Brain graphs are mapped to a low-dimensional manifold using the first 3 components of the normalized graph Laplacian.
  2. Spectral Graph Transformer (SGT): Inspired by Spatial Transformer Networks and PointNet, the SGT learns the transformation matrix required to align a brain's spectral coordinates. To save compute, it takes a random sub-sample of just $N$ nodes in the spectral domain. It passes these through point-wise linear transformations and Multi-Layer Perceptrons (MLPs) to predict a 3x3 orthogonal transformation matrix.
  3. Graph Convolutional Network (GCN): The newly aligned spectral coordinates are fed into a 4-layer GCN equipped with Gaussian kernels. This network classifies each node into specific brain regions (parcels).
  4. End-to-End Training: The SGT and GCN are trained jointly. The loss function combines a spectral alignment loss (Mean Squared Error against the traditional iterative method + an orthogonality constraint) and a parcellation loss (Cross-entropy + Dice loss).

Datasets

  • Mindboggle Dataset: The model is trained and evaluated on 101 manually-labeled cortical meshes. Each brain mesh contains between 102K and 185K vertices, categorized into 32 manually labeled parcels. The dataset was randomly split into 70% training, 10% validation, and 20% testing.

Key Insights and Ideas

  • Data-driven alignment is viable and fast: Instead of treating spectral alignment as a rigid, purely mathematical preprocessing step, it can be learned as a differentiable module within a neural network.
  • Less is more for spatial understanding: The SGT does not need the entire brain mesh to figure out how to align it. The authors found that randomly sampling just 1,000 nodes (out of ~150,000+) is the sweet spot. It provides enough geometric context to capture the global shape of the embedding while keeping memory and compute costs incredibly low.
  • Task-specific alignment beats generic alignment: Training the SGT end-to-end with the GCN yields better parcellation results than training the SGT independently. This proves that joint training allows the network to learn a transformation matrix that is specifically optimized for the downstream task.

Key Results

  • Massive Speedup: The SGT aligns a brain in just 10.7 milliseconds, compared to 15 seconds for the traditional iterative Procrustes method. This is a staggering 1400-fold speedup.
  • High Accuracy: The end-to-end SGT+GCN model achieves an average Dice overlap of 83.26% and an average Hausdorff distance of 1.85 mm.
  • Ablation Performance: Using no alignment at all results in a poor Dice score of 78.82% with irregular parcel boundaries. The SGT recovers nearly all the performance of the slow, traditional iterative alignment (which scores 84.42%).

Limitations and Future Work

  • Limitations: While the SGT is exponentially faster, its parcellation accuracy (83.26% Dice) still slightly underperforms the computationally expensive traditional iterative alignment (84.42% Dice).
  • Future Work: The authors suggest that this fast spectral alignment framework is not limited to parcellation. It can potentially be applied to other surface analysis tasks, such as disease classification or the discovery of new geometry-related biomarkers in the brain.

Summary updated

Abstract

The analysis of the brain surface modeled as a graph mesh is a challenging task. Conventional deep learning approaches often rely on data lying in the Euclidean space. As an extension to irregular graphs, convolution operations are defined in the Fourier or spectral domain. This spectral domain is obtained by decomposing the graph Laplacian, which captures relevant shape information. However, the spectral decomposition across different brain graphs causes inconsistencies between the eigenvectors of individual spectral domains, causing the graph learning algorithm to fail. Current spectral graph convolution methods handle this variance by separately aligning the eigenvectors to a reference brain in a slow iterative step. This paper presents a novel approach for learning the transformation matrix required for aligning brain meshes using a direct data-driven approach. Our alignment and graph processing method provides a fast analysis of brain surfaces. The novel Spectral Graph Transformer (SGT) network proposed in this paper uses very few randomly sub-sampled nodes in the spectral domain to learn the alignment matrix for multiple brain surfaces. We validate the use of this SGT network along with a graph convolution network to perform cortical parcellation. Our method on 101 manually-labeled brain surfaces shows improved parcellation performance over a no-alignment strategy, gaining a significant speed (1400 fold) over traditional iterative alignment approaches.
BibTeX
@inproceedings{he2020spectral,
  author = {Ran He and Karthik Gopinath and Christian Desrosiers and Herve Lombaert},
  title = {{Spectral Graph Transformer Networks for Brain Surface Parcellation}},
  booktitle = {ISBI},
  year = {2020},
  doi = {10.1109/isbi45749.2020.9098737},
}

Loading related content...

Figure