A mixture-of-experts layer splits one large layer into many smaller sub-networks, the "experts," and adds a small model, the router, that decides which experts process each image patch. Sparse Mixture-of-Experts are Domain Generalizable Learners (GMoE) found that such vision transformers generalize better to unseen visual domains, such as training on photos and testing on sketches. Its explanation is that the router learns to send the same object part to the same expert regardless of rendering style. That router, however, reads raw patch features, so it can see global appearance (palette, contrast, illumination) as easily as it sees content, and nothing in the architecture stops it from routing on appearance. The style-transfer literature already offers a closed-form way to remove exactly this information. Adaptive instance normalization (AdaIN), from Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization, shows that each channel's mean and standard deviation, computed across the spatial positions of one image, carry its "style," and that subtracting and dividing them out leaves the layout.
We propose to apply that normalization only on the router's input. The router decides where each patch goes using style-stripped features, while the experts still compute on the full, unnormalized features. Routing then becomes invariant, by construction, to any per-channel rescaling and shift of an image's features, while the representation keeps all of its information. The change is one line of code. The proposal's real contribution is a controlled answer to a question nobody has tested: when a mixture-of-experts model must generalize across domains, should the router see content, style, or both?
Core Claim
Let $X \in \mathbb{R}^{N \times d}$ be the $N$ patch tokens of one image entering a mixture-of-experts layer, after the usual layer norm. Let $\mu, \sigma \in \mathbb{R}^d$ be each channel's mean and standard deviation over the $N$ patches of that image. The style-blind router computes its scores from
$$r_n = \frac{x_n - \mu}{\sigma}, \qquad g(x_n) = \text{Top-}k\left(\text{Softmax}\left(\frac{r_n \cdot e_i}{\|r_n\|\,\|e_i\|}\right)\right),$$
where $x_n$ is token $n$, $e_i$ is the learned embedding of expert $i$ in GMoE's cosine router, and Top-$k$ keeps the $k$ highest-scoring experts. The selected experts then process the original $x_n$, not $r_n$.
Replacing every token by $a \odot x_n + b$, with a per-channel positive scale $a$ and shift $b$ shared across the image, leaves $r_n$ and therefore every routing decision unchanged. This is not redundant with layer norm. Layer norm standardizes each token across its own channels; it does not remove an offset that every patch of the image shares in the same channel. We claim that this exact invariance, applied to routing alone, improves out-of-domain accuracy over GMoE. We also predict that the gain will be largest where shifts are mostly appearance, for example TerraIncognita's illumination changes, and smallest where shifts change shape, for example DomainNet's quickdraw domain. That second prediction follows from the known limit that channel statistics cannot capture geometric change (Multimodal Unsupervised Image-to-Image Translation).
Relation to Prior Work
GMoE motivates the work: it shows routing matters for domain generalization but leaves style-invariance of routing to optimization. Improving Multi-Task Generalization via Regularizing Spurious Correlation (MT-CRL) also motivates it. MT-CRL shows that mixture-of-experts routers latch onto spurious, non-causal features and counters this with decorrelation losses and invariance penalties across environments, which require environment labels. Our constraint is architectural and needs no labels.
Two lines of work go the opposite direction and route on appearance. HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization trains a separate encoder whose discovered latent domains cluster by factors like illumination, and routes on them. CondConv mixes convolution kernels using globally average-pooled features, which are essentially the channel means. The "obvious" bridge between AdaIN and mixture-of-experts would push further in that direction, routing on $[\mu, \sigma]$. We include that design, but as a contrasting arm rather than the proposal, because routing on style tends to create domain-specific experts, and those experts have nothing calibrated to say about a domain never seen in training.
The closest hybrids do not test routing invariance. Dynamic Instance Normalization for Arbitrary Style Transfer generates convolution kernels from style. MoEGen generates adapter weights from mixed expert codes. CARE-Edit routes diffusion tokens to a style-reference expert. All three use style as a signal to inject; none removes it from the routing decision. AdaIN-based style augmentation, as in DCAN and SHADE, supplies the method for our diagnostic and for our strongest competing baseline.
Research Program
The program begins with a diagnostic, because the proposal only makes sense if current routers are in fact sensitive to style.
We train GMoE-S/16 under its published DomainBed protocol, with sparse layers in the last two blocks. We then measure the route-flip rate: the fraction of patches whose top-$k$ expert set changes when the features entering a routing layer are re-styled with AdaIN using the $\mu, \sigma$ of an image from another training domain. We measure this rate both within the training domains and toward the held-out domain, and correlate it per image with misclassification on the held-out domain. CUB-DG, whose stylized bird domains GMoE used to show that part-to-expert assignment is consistent, gives a fine-grained check of whether flips concentrate on background patches or on object parts.
If GMoE's route-flip rate is already near zero, or uncorrelated with error, the core claim is wrong and we stop, reporting the diagnostic as the finding.
If the diagnostic shows real sensitivity, we build three routers that share the same experts, placement, and training recipe. The first routes on raw tokens (GMoE). The second is the style-blind router routing on $r_n$. The third routes on $[\mu, \sigma]$ alone. Comparing the three answers the content-versus-style question directly.
We then probe whether a stronger form of invariance helps. Pixel-level style changes do not map to exact affine changes deep in the network, so we check whether invariance at the routing layer survives input-level re-styling. We also test adding the invariance at the expert inputs, which turns the method into full instance normalization of the layer.
Baselines
The naive reference is a dense ViT-S/16 trained with plain empirical risk minimization, meaning ordinary average-loss training. GMoE reports 47.3% on DomainNet for this model. It tells us whether any routing helps at all.
The strongest directly comparable method is GMoE itself. It reports 48.7% DomainNet accuracy under plain training and 49.6% combined with SWAD, which averages weights along training to find flat minima. The file gives no per-dataset GMoE numbers for the other DomainBed sets, so we rerun GMoE ourselves rather than quote them. We use the same backbone, expert count, top-$k$, training steps, DomainBed hyperparameter search budget, and three seeds per configuration.
Beating GMoE means a higher average across PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet, by more than the seed standard deviation and at least half a point. The gain must also persist when both models are combined with SWAD; anything less suggests the effect is an optimization artifact that weight averaging already captures.
The strongest overall recipe is ERM++. It reports a rise from 61.1% to 78.6% average across five DomainBed datasets on ViT-B/16 with DINOv2 initialization, purely through better training choices. That result uses a different backbone and initialization, so we do not claim to beat it at ViT-S. Instead we run the style-blind router on top of the ERM++ recipe at ViT-B/16 to see whether the gain survives a strong initialization.
The ablation that isolates the key ingredient is style augmentation applied to GMoE. During training, AdaIN swaps channel statistics between images at the routing layers, in the spirit of DCAN's feature alignment and SHADE's style hallucination, so that the raw-token router can learn invariance rather than be given it. A second ablation is the statistics-only router, which tests the opposite hypothesis that HMOE and CondConv implicitly embody.
Why not just…?
Why not just tune plain training better, as ERM++ and SWAD do? These recipes give large gains without any architectural change. GMoE's own results nonetheless show that sparse routing adds on top of SWAD (48.7% to 49.6% on DomainNet), so the two are complementary rather than competing. Our SWAD-combined and ERM++-recipe runs test whether that complementarity holds for the style-blind router specifically. If the gain vanishes under ERM++, we will report that the router change only matters for weaker training recipes.
Why not just train GMoE with AdaIN style augmentation so the router learns invariance itself? This is the real threat, since DCAN and SHADE show that statistic-swapping augmentation is a strong, cheap tool. Our argument is that augmentation only covers the styles sampled from training domains, while normalizing the router input covers the entire family of per-channel affine changes, including ones never sampled. SWAD also cautions that some augmentations help in-domain but hurt out-of-domain. The augmented-GMoE ablation, plus the route-flip rate measured toward the held-out domain, settles this. If augmentation matches our method on both, the honest conclusion is that augmentation suffices and the architectural constraint is unnecessary.
Why not route on the channel statistics themselves, since they capture domain for free? HMOE's latent domains cluster by illumination, which makes this tempting. But experts specialized by training domain must extrapolate on a novel one. The statistics-only arm tests this directly, and we expect it to help in-domain and to hurt on held-out domains.
Why not instance-normalize the whole layer, or rely on layer norm already present? Layer norm does not remove image-wide per-channel offsets, so it gives no routing invariance. Full instance normalization discards global statistics that may carry class information, such as a dominant color. Our expert-input ablation measures that cost.
Decisive Checks
The proposal stands or falls on three outcomes, in order. First, the diagnostic must show that GMoE routes change under cross-domain feature re-styling and that those changes track errors on the held-out domain. Second, the style-blind router must beat both GMoE and style-augmented GMoE on the DomainBed average, with and without SWAD. Third, the gains must be ordered by how much each shift is appearance rather than shape: largest on TerraIncognita and the stylized CUB-DG domains, smallest on DomainNet quickdraw.
A result where the style-blind router wins overall but its gains do not follow that ordering would mean the mechanism is something other than style invariance. We would report it that way.
References
- Sparse Mixture-of-Experts are Domain Generalizable Learners
- Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization
- Multimodal Unsupervised Image-to-Image Translation
- Improving Multi-Task Generalization via Regularizing Spurious Correlation
- HMOE: Hypernetwork-based Mixture of Experts for Domain Generalization
- CondConv: Conditionally Parameterized Convolutions for Efficient Inference
- Dynamic Instance Normalization for Arbitrary Style Transfer
- MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation
- CARE-Edit: Condition-Aware Routing of Experts for Contextual Image Editing
- DCAN: Dual Channel-Wise Alignment Networks for Unsupervised Scene Adaptation
- Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation
- SWAD: Domain Generalization by Seeking Flat Minima
- ERM++: An Improved Baseline for Domain Generalization