Rethinking MoE Load Balancing as Feedback Control

Yikun Hou, Alp YurtseverarXiv 2026 · arXiv

TL;DR

Load balancing in Mixture-of-Experts (MoE) models often breaks down when the data distribution shifts during training. By reframing load balancing as a classical feedback control problem, this paper introduces PID (I)—a simple, linear integral controller that maintains expert balance more effectively than existing methods without sacrificing model performance.

What Problem Does This Paper Address?

Mixture-of-Experts (MoE) architectures are powerful because they decouple model capacity from compute cost by routing tokens to a sparse subset of "experts." However, they suffer from a "rich-get-richer" dynamic: early-trained experts attract more tokens, receive more gradient updates, and become even better, while others are starved and left undertrained.

Existing load-balancing mechanisms (like auxiliary losses or per-expert biases) were designed for stationary data. But real-world training isn't stationary—data mixtures change at curriculum boundaries, during continual pretraining, or when a corpus is refreshed. These shifts disrupt the router's equilibrium, leading to "load collapse." The authors seek a balancer that can not only hold balance during steady states but also rapidly restore it after a distribution shift.

Methods

The authors rethink load balancing as a feedback control loop. In this view:

  • Error ($e$): The deviation of an expert's current load from its "fair share."
  • Control Input ($u$): A per-expert bias added to the router scores.
  • Plant: The router itself, which maps biases to expert loads.

They define the PID-Balancer, a family of controllers based on Proportional (P), Integral (I), and Derivative (D) terms. Crucially, they show that existing popular methods are actually specific members of this family:

  • Auxiliary Loss: Behaves like a Proportional controller.
  • ALF-LB (Auxiliary-Loss-Free): Acts as a Relay-Integral controller (using the sign of the error).

Through control analysis, they find that the measured load at each step is dominated by high-frequency "micro-batch noise." Proportional and derivative terms tend to amplify this noise into the routing bias. In contrast, the Integral (I) term—which accumulates past errors—acts as a low-pass filter, averaging out noise while effectively removing persistent imbalance. Consequently, they propose PID (I), a pure linear integral controller with a single gain parameter ($K_I$).

Datasets

  • SlimPajama-627B: Used to create domain-extracted subsets (ArXiv, Books, C4, GitHub, etc.) for pretraining experiments with controlled domain shifts (e.g., switching from Books to GitHub mid-run).
  • OLMoE-1B-7B: A released MoE checkpoint used for continual pretraining experiments.
  • CodeParrot: Used as the target corpus for continual pretraining.
  • Evaluation Benchmarks: MMLU, HellaSwag, ARC-Challenge, HumanEval, and MBPP were used to ensure task performance was maintained.

Key Insights and Ideas

  • The Power of Memory: The integral term is the only one with "memory." This allows it to distinguish between a random spike in tokens for an expert (noise) and a persistent trend of over-utilization (imbalance).
  • Bias vs. Gradient: Bias-based balancers (like PID (I)) are "drop-in" solutions. Unlike auxiliary losses, they don't modify the model's weights or the training objective directly. This allows experts to retain their learned domain specialization in the gate weights even while the bias forces a balanced selection at runtime.
  • Noise Sensitivity: The paper highlights that reacting to step-to-step fluctuations is "wasted effort" because the bias acts one step late. By the time the bias adjusts to a noisy batch, that batch is gone. The integral term's averaging property is the principled way to handle this.

Key Results

  • Superior Balance: Across all scales (16 to 64 experts), PID (I) achieved the lowest MaxVio (maximum overload). As seen in Figure 1, PID (I) consistently settles at a lower post-shift imbalance than ALF-LB or Auxiliary Loss.
Figure 1: Pre- and post-shift MaxVio for the three pre-shift scenarios of Section 4.1. The circle marks the transition from the pre-shift steady state (preMV) to the post-shift steady state (postMV).
  • Rapid Recovery: When a distribution shift occurs, PID (I) restores balance more tightly and quickly. Figure 2 illustrates how PID (I) (green line) maintains a much lower and steadier load violation throughout the training process compared to the volatile vanilla baseline or the less effective auxiliary loss.
Figure 2: MaxVio over training (E = 64, 1 →1 scenario, 6-seed mean, 51-step smoothing). Shading marks the 2,000-step windows averaged for PreMV and PostMV.
  • No Performance Hit: The improved balance comes at no cost to language modeling quality. Perplexity (PPL) and downstream benchmark scores remained comparable across all balancing methods (see Table 1).
Table 1: Post-shift steady-state comparison across expert count E ∈{16, 32, 64} and the three pre-shift mixture scenarios of Section 4.1. Arrows show the better direction (↑higher, ↓lower).

Limitations and Future Work

  • Scale: The experiments were conducted on models up to 7B parameters. Whether these findings hold for "production-scale" MoEs (trillions of parameters) remains an open question.
  • Routing Architectures: The study focused on Top-K routing. The authors suggest that exploring how this feedback control approach applies to other designs, like expert-choice routing or routers with hard capacity limits, is a promising future direction.
  • Error Definition: The current PID (I) uses a per-step error. The authors hypothesize that a smoothed error definition might eventually allow the Proportional and Derivative terms to contribute useful signals, potentially enabling the use of the full PID family.

Summary updated

Abstract

Mixture-of-Experts (MoE) training is often not stationary: the data mixture changes at curriculum boundaries, when a model is continually pretrained into a new domain, or when the corpus is refreshed. Routing is learned against the current mixture, so each such change disrupts the per-expert load the router had converged to, and existing balancing methods have only been evaluated on stationary data. We rethink load balancing as a feedback control problem, in which the deviation of each expert's load from its fair share is the control error and the per-expert routing bias is the control input, and define a family of controllers, the PID-Balancer, from the proportional, integral, and derivative terms of classical PID control. Two widely used methods belong to this family: the auxiliary balancing loss behaves like a proportional controller, and the loss-free bias update is an integral controller acting on the sign of the error. Reading the router as a static map with a one-step delay, we show that the integral term is the only one that both removes persistent imbalance and averages out the step-to-step fluctuation of the measured load, whereas the proportional and derivative terms pass or amplify that fluctuation into the routing bias. We therefore adopt PID (I), the pure-integral member of the family with a single gain. Two settings are evaluated: (i) training an OLMoE backbone from scratch with 16 to 64 experts per layer, through a mid-run mixture switch, and (ii) continual pretraining of the released OLMoE-1B-7B on a new corpus. PID (I) holds balance before the shift and restores it more tightly afterwards than either the auxiliary loss or loss-free balancing, at comparable perplexity.
BibTeX
@misc{hou2026rethinking,
  author = {Yikun Hou and Alp Yurtsever},
  title = {{Rethinking MoE Load Balancing as Feedback Control}},
  year = {2026},
  eprint = {2611.09252},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09252},
}
Figure