TL;DR
The authors propose GNNMoE, a framework that applies the "Mixture of Experts" (MoE) architecture to Graph Neural Networks to solve the "one-size-fits-all" limitation of standard message passing. By using an entropy-driven routing mechanism, the model dynamically learns whether a specific node needs to aggregate information from its neighbors (homophily) or rely more on its own features (heterophily), consistently outperforming state-of-the-art methods across diverse graph types.
What Problem Does This Paper Address?
Most classical Graph Neural Networks (GNNs) like GCN or GAT operate on the homophily assumption: the idea that connected nodes are likely to share the same labels or features (e.g., friends in a social network often share interests). However, real-world graphs are often heterophilous, where connected nodes are different (e.g., a fraudster connecting to many legitimate accounts).
Standard GNNs use a uniform message-passing scheme across the entire graph. This is suboptimal because, as shown in Figure 1, a single graph can contain a mix of neighborhood contexts. Some nodes are in highly consistent clusters, while others sit at the intersection of diverse communities.

Existing attempts to fix this often rely on static rules or coarse-grained partitioning. The challenge is creating a model that can adapt its learning strategy at the individual node level without massive computational overhead.
Methods
The core of the paper is GNNMoE, which breaks down the GNN process into specialized "experts" and a smart "router."
- Message Passing Decomposition: The authors decompose message passing into two operations: Propagation (P) (smoothing features across neighbors) and Transformation (T) (applying non-linear weights to a node's own features).
- The Experts: GNNMoE uses four expert networks representing different combinations of these operations: PP, PT, TP, and TT.
- Hybrid Routing Architecture:
- Soft Routing: Each node's features are passed through a routing layer that assigns weights to the four experts.
- Enhanced Feed-Forward Network (EFFN): After the MoE blocks, the model uses a hard routing layer to choose the best activation function (SwishGLU, GEGLU, or REGLU) to further refine the representation.
- Routing Entropy Regularization: This is the "secret sauce." Instead of manually picking how many experts to use (top-$k$), the authors introduce a regularization term based on entropy.
- High entropy results in "soft weighting" (using many experts), which is better for complex, heterophilous nodes.
- Low entropy results in "soft top-$k$" (focusing on one or two dominant experts), which is better for simple, homophilous nodes.

Datasets
The model was evaluated on 12 benchmark datasets to test its versatility:
- 6 Homophilous Graphs: Computers, Photo, Coauthor CS, Coauthor Physics, Facebook, and Ogbn-arxiv.
- 6 Heterophilous Graphs: Actor, Squirrel-fix, Chameleon-fix, Tolokers, Roman-empire, and Penn94.
These datasets cover a wide range of domains, from citation networks to social media and Wikipedia page-links.
Key Insights and Ideas
- Nodes have distinct "tastes": The authors empirically proved that high-homophily nodes prefer propagation-heavy schemes (PP, PT), while low-homophily nodes favor transformation (TT).
- Graph-level preferences: Homophilous graphs generally perform better with "hard" routing (picking the single best expert), whereas heterophilous graphs need "soft" routing (blending multiple experts) to handle their inherent noise and diversity.
- Entropy as a Bridge: By controlling the entropy of the routing weights, GNNMoE can automatically transition between these two strategies (soft vs. hard routing) during training, making it truly adaptive to the specific graph it is processing.
Key Results
GNNMoE demonstrated significant performance gains over 20 baseline models, including vanilla GNNs, specialized heterophilous GNNs, and Graph Transformers.
- Accuracy: GNNMoE achieved the best performance on 11 out of 12 datasets.
- Relative Gains: It showed improvements of up to 20.47% over vanilla GNNs and 12.06% over other specialized heterophilous models.
- Efficiency: Despite the complexity of having multiple experts, GNNMoE is highly efficient. On large datasets like Ogbn-arxiv, it was 2 to 7 times faster than many Graph Transformer competitors and avoided the Out-of-Memory (OOM) issues that plague global attention mechanisms.
Limitations and Future Work
While GNNMoE is a powerful step forward for node classification, the authors note a few areas for growth:
- Complex Graph Tasks: The current work focuses on node classification; future research could extend this MoE approach to link prediction or graph-level classification.
- Theoretical Bounds: While the entropy regularization works well in practice, deeper theoretical exploration of the bounds of entropy-driven routing could yield even more robust models.
- Broader Graph Types: Testing the framework on more complex structures, such as hypergraphs or dynamic graphs, remains an open direction.
Summary updated