Routing Traces as a Readout of What a Mixture-of-Experts Model Knows

A mixture-of-experts (MoE) language model replaces each dense feed-forward block with several parallel sub-networks, called experts. A small learned gate, the router, sends each token to only a few of them. Every forward pass therefore produces, as a side effect, a record of which experts handled which token at which layer. This routing trace is tiny. For Mixtral of Experts, with 32 layers and 2 of 8 experts chosen per layer, it comes to about 150 bits per token, against more than a hundred thousand floating-point numbers of hidden state. Serving systems already compute the trace in order to dispatch tokens. We propose treating it as a readout of the model's internal state. We ask whether a model's routing alone reveals when it is about to state a false fact. Then we use the experts that carry that signal as the place to intervene, only on the tokens where the signal fires.

This matters for two reasons. Error detection from hidden states works well: LLMs Know More Than They Show trains linear probes on internal activations at the answer token and reports AUC of 0.85 to 0.95. But such probes need full white-box access, which rules them out for anyone who is not running the weights. A routing trace is a much narrower channel that a model provider could expose or log without revealing weights or activations. Separately, the MoE steering literature intervenes on routers but chooses which experts to push using static contrastive statistics, and applies the push to every token. If one compact signal can both detect a likely error and name the experts to suppress, detection and control collapse into the same object.

Core Claim

Our claim has three parts. First, a probe trained only on per-layer router probabilities retains most of the factual-error signal that a hidden-state probe finds, at roughly three orders of magnitude less bandwidth. Second, how much it retains grows with the number of experts, as semantic routing becomes more pronounced. Third, the experts that the routing probe weighs most heavily are causal: suppressing them only on flagged tokens improves correctness and context faithfulness more than always-on expert suppression, at lower cost to general capability.

We are explicit that routing cannot contain more information than the hidden states. Router logits are a linear map of the residual stream, so a probe over all hidden layers upper-bounds a probe over all routers. The claim is therefore about how cheaply the signal can be obtained and whether it points to a place to act, not about beating hidden-state probes on accuracy.

Three lines of prior work motivate this. Probing Semantic Routing in Large Mixture-of-Expert Models shows that large MoEs, with 128 to 256 experts, route by meaning rather than surface token identity. It also shows that uncertainty words route to dedicated expert subsets. However, it is purely correlational and never predicts correctness from routing. Understanding Cross-layer Contributions to Mixture-of-Experts Routing in LLMs shows that early "kingmaker" experts shape routing 10 to 15 layers later. That suggests the whole-depth trace carries structured information that no single layer shows, but the paper stops at diagnosis. Understanding Multilingualism in Mixture-of-Experts LLMs finds that middle layers act as language-agnostic hubs. This is where we expect the task-general part of the signal to live.

Research Program

The readout is simple. Let $g_l(x_t) \in \mathbb{R}^E$ be the router's softmax distribution over $E$ experts at layer $l$ for token $x_t$. We define the trace $r_t = [g_1(x_t); \dots; g_L(x_t)]$, which has $L \times E$ numbers: 256 for Mixtral. We also define a discrete variant that keeps only the top-$k$ indices. The error probe is

$$p(\text{error} \mid x_t) = \sigma(w^\top r_t + b),$$

where $\sigma$ is the logistic function and $w, b$ are fit by logistic regression. We read $r_t$ at two positions. The first is the exact answer token, following Orgad et al. The second is the first generated token, following On Early Detection of Hallucinations in Factual Question Answering, which predicts errors before a claim is emitted.

The analysis stage asks where the signal lives. We fit probes on single layers, on contiguous layer bands, and on the full trace. We compare full gate probabilities against top-$k$ indices only, since indices are what a serving system logs for free. We run the same pipeline on three models with increasing expert counts: Mixtral 8x7B (8 experts), DeepSeekMoE-16B (a fine-grained MoE with shared and routed experts, used by AEA), and GPT-OSS-120B, which SteerMoE also studies. This tests the prediction from Olson et al. that semantic routing strengthens with expert count.

For DeepSeekMoE we separate shared experts from routed ones. Decoding Knowledge Attribution in Mixture-of-Experts argues that shared experts carry general patterns while routed experts refine niche details. That predicts the factual-error signal concentrates in routed experts, which is where a router-level intervention can reach.

The intervention stage turns probe weights into an action. We pick the set $S$ of experts with the largest positive probe weights, meaning those whose selection predicts error. At generation time, if $p(\text{error}) > \tau$, we subtract a margin $m$ from those experts' router logits, both at the current token and for a short window afterward. Otherwise the model runs untouched. Two things distinguish this from Steering MoE LLMs via Expert (De)Activation. SteerMoE ranks experts by a risk-difference score over contrastive response pairs and applies the margin on every token. We select experts by their contribution to a correctness predictor and apply the margin conditionally.

We keep SteerMoE's reported finding that deactivating experts is gentler than forcing them on. This is why we only ever suppress. Because Li et al. show that early experts reshape later routing, we also measure how much each intervention perturbs downstream routing, using their Average Absolute Ranking Variation (AARV) metric. This checks whether suppressing a middle-layer expert is undone or amplified deeper in the network.

Baselines

The naive reference points are the unmodified model and a black-box confidence signal. For detection, that signal is the mean token log-probability and entropy of the generated answer. If routing does not beat log-probability, the trace adds nothing beyond what any API already exposes. For intervention, the reference is greedy decoding, plus the same model abstaining whenever log-probability confidence is low.

The strongest detection competitor is the exact-answer-token hidden-state probe of Orgad et al., which reports AUC between 0.85 and 0.95 on TriviaQA, HotpotQA, and Natural Questions. Those numbers come from dense Mistral-7B and Llama-3-8B. For hidden-state probes on MoE models we found no reported result on Lacuna, so we will run this probe ourselves on each MoE. We run it at the same token positions and on the same train and test splits, with the same regularization search, both at the best single layer and across all layers concatenated.

We will say routing "keeps most of the signal" if it stays within 0.03 AUROC of the best single-layer hidden probe. We will say it beats the practical baseline if it exceeds the log-probability detector by at least 0.05. The first-token probe of Snyder et al., which reports AUROC up to 0.81 on Falcon-40B over T-REx and TriviaQA, serves as the early-detection comparison.

The strongest intervention competitor is SteerMoE, which reports up to a +27% faithfulness improvement on RAG tasks including FaithEval. We will rerun it on our models with its risk-difference expert selection. It gets the same expert budget $|S|$ and the same margin grid as our method, with $m$ and $|S|$ tuned on the same held-out split. Our method wins if it matches SteerMoE's FaithEval gain while losing less on MMLU and MT-Bench, or if it exceeds the gain at equal utility cost.

The key ablation keeps everything fixed but applies our probe-selected experts on every token. This separates the effect of which experts are chosen from the effect of gating by the probe. A second ablation uses randomly selected experts under the same gate. A third replaces router suppression with Contrastive Activation Addition, which adds a mean-difference direction to the residual stream, under the same probe gate. This tests whether acting on the router is better than acting on the continuous state.

Why not just…?

Why not just use a hidden-state probe, since it provably contains at least as much information? For white-box users, a hidden-state probe may be all they need, and we expect it to win on raw AUROC. The routing readout matters where hidden states are unavailable or too expensive to store per token. It also matters because a probe over experts yields an intervention target directly, whereas a hidden-state probe yields a direction that must then be turned into a steering vector. If the gap turns out to exceed 0.03 AUROC on the larger models, the monitoring claim fails. We would then report routing only as an intervention-selection tool.

Why not just threshold output log-probability, which any API already returns? Orgad et al. and Do Androids Know They're Only Dreaming of Electric Sheep? both find that internal states detect errors better than output-level signals. Orgad et al. also warn that many "truth" probes merely capture output uncertainty. We test this directly in two ways. We report the routing probe's AUROC after including log-probability as a covariate. We also evaluate on items where log-probability is high but the answer is wrong. If routing adds nothing there, it is only a proxy for uncertainty.

Why not just run SteerMoE always-on, or abstain whenever the probe fires? Always-on suppression is the real threat to the intervention half. If the gated and ungated versions of our experts perform the same, gating is pointless and the contribution reduces to a different expert-selection rule. The first ablation above settles this.

Abstention is a legitimate alternative for detection. But it cannot recover correct answers, and Orgad et al. show that correct answers are often recoverable: reranking sampled candidates by a probe raised accuracy by 30 to 40 points on queries the model consistently failed. We therefore report accuracy on the flagged subset, not just the error rate. We also include a routing-probe candidate reranker alongside router suppression.

Why not amortize the fix into the weights as CASAL does? CASAL trains a single sub-module to reproduce steered activations and reports 30 to 40% hallucination reduction with little data. It treats an MoE like a dense model, without asking which expert it edits. This is complementary to our method rather than a replacement. If the routing probe localizes the error to specific routed experts, CASAL could be applied to exactly those experts. We treat this as a follow-up once localization is established.

Decisive Checks

For detection, we evaluate on TriviaQA, Natural Questions, and HotpotQA in closed-book form, and on PopQA to stress rare entities. Metrics are AUROC and the bits per token of the readout. The central confound is topic. Routing plausibly encodes domain, and some domains are simply harder. So we report within-domain AUROC by stratifying on question category and entity popularity. We also include a control probe on question bag-of-words features. A routing probe that falls to the control's level after stratification would refute the claim.

Two further checks probe the claim's scope. The first is transfer. Orgad et al. show hidden-state truth probes fail to move from trivia to MATH. We test whether a probe restricted to the middle "semantic hub" layers transfers from TriviaQA to GSM8K and MATH better than a hidden-state probe does. Either outcome is informative. The second is the scaling curve across the three models. Olson et al.'s findings predict that the routing-to-hidden AUROC gap narrows as expert count grows.

For intervention, the main benchmark is FaithEval across its unanswerable, inconsistent, and counterfactual splits, where SteerMoE's reported gain gives a concrete target. We add closed-book TriviaQA accuracy and TruthfulQA judged by GPT-judge. For TruthfulQA, DoLa reports roughly 12 to 17 absolute points of improvement on LLaMA models, so we include it as a decoding-side comparison. Utility is measured on MMLU and MT-Bench. We also track the fraction of tokens intervened on, and downstream routing disruption via AARV.

The proposal succeeds if three things hold together: the routing probe comes within 0.03 AUROC of hidden-state probes on the largest model, it clearly beats log-probability after the topic controls, and probe-gated suppression beats both always-on suppression and random-expert gating on the faithfulness-utility trade-off. If any of the three fails, the experiments tell us which half of the idea (monitoring or control) survives.