Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts

Lean Wang, Huazuo Gao, Chenggang Zhao, Xu Sun, Damai Dai

Abstract: For Mixture-of-Experts (MoE) models, an unbalanced expert load will lead to routing collapse or increased computational overhead. Existing methods commonly employ an auxiliary loss to encourage load balance, but a large auxiliary loss will introduce non-negligible interference gradients into training and thus impair the model performance. In order to control load balance while not producing undesired gradients during training, we propose Loss-Free Balancing, featured by an auxiliary-loss-free load balancing strategy. To be specific, before the top-K routing decision, Loss-Free Balancing will first apply an expert-wise bias to the routing scores of each expert. By dynamically updating the bias of each expert according to its recent load, Loss-Free Balancing can consistently maintain a balanced distribution of expert load. In addition, since Loss-Free Balancing does not produce any interference gradients, it also elevates the upper bound of model performance gained from MoE training. We validate the performance of Loss-Free Balancing on MoE models with up to 3B parameters trained on up to 200B tokens. Experimental results show that Loss-Free Balancing achieves both better performance and better load balance compared with traditional auxiliary-loss-controlled load balancing strategies.

Introduction

Mixture-of-Experts (MoE) architectures have become a cornerstone for scaling Large Language Models (LLMs), enabling massive parameter counts with sparse activation to keep computational costs manageable. However, a persistent challenge in MoE training is load imbalance. Without intervention, routers tend to converge on a subset of experts, leading to "routing collapse" where most experts remain underutilized.

Traditionally, this is addressed by adding an auxiliary loss to the training objective. While effective at balancing load, this approach introduces "interference gradients." These gradients force a trade-off between the primary language modeling objective and the secondary balancing objective, often degrading final model performance.

The Loss-Free Balancing Approach

To resolve this trade-off, researchers from DeepSeek-AI and Peking University developed Loss-Free Balancing (LFB). Instead of modifying the loss function, LFB integrates balancing logic directly into the routing mechanism.

Overview of the Loss-Free Balancing mechanism during model training and bias updating.

The core of LFB lies in three components:

  1. Biased Gating Scores: An expert-wise bias ($b_i$) is added to the raw routing scores before the Top-K selection process.
  2. Iterative Bias Updating: These biases are not learned via backpropagation. Instead, they are updated after each training step based on the observed load of the previous batch. If an expert is overloaded, its bias decreases; if underloaded, its bias increases.
  3. Selection vs. Weighting: Crucially, the bias is only used for expert selection. The final weights used to combine expert outputs remain a pure function of the learned features, ensuring that the balancing heuristic does not distort the model's representational output.

Experimental Results

The method was evaluated using a large-scale multilingual corpus on models up to 3B parameters. LFB consistently outperforms traditional auxiliary-loss methods across both performance and balance metrics.

Comparison of Perplexity vs. Load Balance (MaxVio) for Loss-Controlled and Loss-Free strategies.

As shown in the figure above, LFB (the red star) achieves a superior Pareto frontier. While traditional methods (blue dots) must sacrifice perplexity to achieve better balance (lower $\alpha$ values), LFB achieves the lowest perplexity and the best balance simultaneously.

Specifically, on the 1B model, LFB achieved a validation perplexity of 9.50 compared to 9.56 for the baseline. More impressively, it reduced the "Max Violation" (imbalance measure) from 0.72 to just 0.04. This demonstrates that by removing interference gradients, the model can focus entirely on language modeling while maintaining near-perfect expert utilization.

Conclusion and Future Work

Loss-Free Balancing represents a significant step forward for MoE training. By decoupling load balancing from the loss objective, it eliminates the performance "tax" associated with auxiliary losses. While future work may explore hyperparameter sensitivity at even larger scales (100B+), the current results provide a compelling case for LFB as a more efficient and effective standard for sparse model training.

Summary updated

Abstract

For Mixture-of-Experts (MoE) models, an unbalanced expert load will lead to routing collapse or increased computational overhead. Existing methods commonly employ an auxiliary loss to encourage load balance, but a large auxiliary loss will introduce non-negligible interference gradients into training and thus impair the model performance. In order to control load balance while not producing undesired gradients during training, we propose Loss-Free Balancing, featured by an auxiliary-loss-free load balancing strategy. To be specific, before the top-K routing decision, Loss-Free Balancing will first apply an expert-wise bias to the routing scores of each expert. By dynamically updating the bias of each expert according to its recent load, Loss-Free Balancing can consistently maintain a balanced distribution of expert load. In addition, since Loss-Free Balancing does not produce any interference gradients, it also elevates the upper bound of model performance gained from MoE training. We validate the performance of Loss-Free Balancing on MoE models with up to 3B parameters trained on up to 200B tokens. Experimental results show that Loss-Free Balancing achieves both better performance and better load balance compared with traditional auxiliary-loss-controlled load balancing strategies.
BibTeX
@misc{wang2024auxiliary,
  author = {Lean Wang and Huazuo Gao and Chenggang Zhao and Xu Sun and Damai Dai},
  title = {{Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts}},
  year = {2024},
  eprint = {2408.15664},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.15664},
}
Figure