Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers

A. Q. Chen, Kaikai An, Xin Zhang, Fang Dong, Hengjie Cao, Zhendong Huang, Yifeng Yang, Mengyi Chen, Jie Zhou, Mingzhi Dong, Yujiang Wang, Jinlong Hou, Qin Lv, Robert P. Dick, Yuan Cheng, Tun Lu, Fan Yang, Li Shang

Abstract: Mixture-of-Experts (MoE) architectures are often considered a natural fit for continual learning because sparse routing should localize updates and reduce interference, yet MoE Transformers still forget substantially even with sparse, well-balanced expert utilization. We attribute this gap to a pre-routing bottleneck: multi-head attention concatenates head-specific signals into a single post-attention router input, forcing routing to act on co-occurring feature compositions rather than separable head channels. We show that this router input simultaneously encodes multiple separately decodable semantic and structural factors with uneven head support, and that different feature compositions induce weakly aligned parameter-gradient directions; as a result, routing maps many distinct compositions to the same route. We quantify this collision effect via a route-wise effective composition number $N_{eff}$ and find that higher $N_{eff}$ is associated with larger old-task loss increases after continual training. Motivated by these findings, we propose MH-MoE, which performs head-wise routing over sub-representations to increase routing granularity and reduce composition collisions. On TRACE with Qwen3-0.6B/8B, MH-MoE effectively mitigates forgetting, reducing BWT on Qwen3-0.6B from 11.2% (LoRAMoE) to 4.5%.

The Pre-Routing Bottleneck

In continual learning, models often suffer from catastrophic forgetting, where learning new tasks overwrites knowledge from previous ones. While Mixture-of-Experts (MoE) architectures were theoretically expected to solve this by localizing updates to specific "experts," practical implementations in Transformers still struggle.

This research identifies the culprit as the Multi-Head Attention (MHA) mechanism. In standard Transformers, the outputs of multiple attention heads—each specializing in different semantic or structural features—are concatenated into a single vector before being sent to the MoE router. This creates a bottleneck where the router must make a single decision for a complex "cocktail" of features. This leads to composition collisions, where distinct features are forced to share the same experts, causing gradient interference.

Diagnosing Composition Collisions

By using linear probes, the authors demonstrate that different features (such as domain identity, token frequency, or stance) are concentrated in specific attention heads. However, because these signals are merged before routing, the router cannot distinguish between them effectively.

Feature importance across attention heads shows that different semantic and structural factors are supported by distinct heads.

To quantify this, the authors introduce the Effective Composition Number ($N_{eff}$), which measures how many different feature combinations are funneled into the same expert. They find that higher $N_{eff}$ values correlate directly with increased forgetting of old tasks.

MH-MoE: Granular Head-Wise Routing

To address this bottleneck, the authors propose MH-MoE (Multi-Head Mixture-of-Experts). The core idea is to move from token-level routing to head-level routing:

  1. Head-Aligned Splitting: The post-attention representation is split back into its original head-specific slices.
  2. Head-Private Routing: Each head is assigned its own independent router and a private bank of experts.
  3. Increased Granularity: By routing at the head level, the model can isolate updates to specific functional components, drastically reducing the likelihood of unrelated features colliding in the same parameter space.

Why It Works: Gradient Alignment

The effectiveness of MH-MoE is rooted in how it manages gradient updates. The study proves that different feature compositions produce gradients that are "weakly aligned" (nearly orthogonal). When these compositions are forced to share an expert in standard MoE, the update for a new task is likely to damage the parameters supporting an old task.

Histogram of gradient cosine similarity showing that different compositions (across-comp) have nearly orthogonal gradients, leading to interference when sharing experts.

Experimental Results

MH-MoE was evaluated on the TRACE benchmark, which includes eight diverse tasks ranging from stance detection to code completion, using Qwen3-0.6B and Qwen3-8B models.

Key findings include:

  • Significant Forgetting Reduction: On Qwen3-0.6B, the Backward Transfer (BWT) improved from -11.2% (using LoRAMoE) to just -4.5%.
  • Improved Performance: Overall Performance (OP) on TRACE increased from 0.378 to 0.467 for the 0.6B model.
  • Efficiency: MH-MoE is a practical drop-in replacement, adding only ~1% training latency and ~4% memory overhead.

By increasing routing granularity to match the inherent structure of Multi-Head Attention, MH-MoE provides a robust, task-agnostic architectural solution to catastrophic forgetting in large-scale MoE models.

Summary updated

Abstract

Mixture-of-Experts (MoE) architectures are often considered a natural fit for continual learning because sparse routing should localize updates and reduce interference, yet MoE Transformers still forget substantially even with sparse, well-balanced expert utilization. We attribute this gap to a pre-routing bottleneck: multi-head attention concatenates head-specific signals into a single post-attention router input, forcing routing to act on co-occurring feature compositions rather than separable head channels. We show that this router input simultaneously encodes multiple separately decodable semantic and structural factors with uneven head support, and that different feature compositions induce weakly aligned parameter-gradient directions; as a result, routing maps many distinct compositions to the same route. We quantify this collision effect via a route-wise effective composition number $N_{eff}$ and find that higher $N_{eff}$ is associated with larger old-task loss increases after continual training. Motivated by these findings, we propose MH-MoE, which performs head-wise routing over sub-representations to increase routing granularity and reduce composition collisions. On TRACE with Qwen3-0.6B/8B, MH-MoE effectively mitigates forgetting, reducing BWT on Qwen3-0.6B from 11.2% (LoRAMoE) to 4.5%.
BibTeX
@inproceedings{chen2026multi,
  author = {A. Q. Chen and Kaikai An and Xin Zhang and Fang Dong and Hengjie Cao and Zhendong Huang and Yifeng Yang and Mengyi Chen and Jie Zhou and Mingzhi Dong and Yujiang Wang and Jinlong Hou and Qin Lv and Robert P. Dick and Yuan Cheng and Tun Lu and Fan Yang and Li Shang},
  title = {{Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=iXhFXwIbvi},
}
Figure