Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR

Chongyu Fan, Gaowen Liu, Mingyi Hong, Ramana Rao Kompella, Sijia LiuarXiv 2026 · arXiv

TL;DR

While the popular matrix-aware optimizer Muon excels at LLM pretraining via uniform spectral whitening, this paper reveals that it fails dramatically in post-pretraining regimes like Vision-Language-Action (VLA) models and Reinforcement Learning with Verifiable Rewards (RLVR). To fix this, the authors introduce Pion, a drop-in replacement that uses a novel "Promotion+Suppression" polynomial mechanism to act as a spectral high-pass filter, suppressing gradient noise while retaining critical directions without adding any computational overhead.

What Problem Does This Paper Address?

Muon has recently made waves in LLM pretraining by departing from traditional element-wise optimization (like AdamW) in favor of a matrix-aware approach. It leverages Newton-Schulz (NS) iterations to drive all singular values of the momentum matrix to 1—a process known as uniform spectral whitening.

While this uniform whitening encourages broad exploration during pretraining, the authors identify a critical "spectral mismatch" when applying Muon to other increasingly important training paradigms:

  1. Cross-Modality VLA Training: Action-module gradients are inherently low-rank compared to high-rank vision and language features.
  2. RLVR Post-Training: Policy gradients suffer from a very low signal-to-noise ratio (SNR).

In both cases, Muon's uniform whitening inflates the noisy "tail" of the singular value spectrum to the same magnitude as the informative "head." This amplifies gradient noise, leading to suboptimal performance in VLA models and catastrophic model collapse in RLVR.

Methods

The authors propose Pion (sPectral hIgh-pass Optimization on momeNtum) as a computationally efficient, drop-in alternative to Muon. Instead of uniformly whitening the gradient spectrum, Pion splits the Newton-Schulz iterations into a two-stage polynomial mapping:

  • Promotion Stage: Monotonically amplifies singular values to lift dominant components.
  • Suppression Stage: Acts as a soft high-pass filter, anchoring leading singular values near 1 while driving noisy tail components toward 0.
Visualization of the high-pass filter design in Pion compared to Muon.

Furthermore, to respect the heterogeneous update scales required across attention heads inherited from pretraining, Pion introduces a per-head mode. This mode reshapes attention projections along the head dimension and applies the high-pass filter independently per head at zero extra computational cost.

Datasets

The authors evaluated Pion across two distinct domains:

  • VLA Training: Evaluated in simulation using the four LIBERO suites (Object, Spatial, Goal, Long) and LIBERO-Plus (which introduces environmental and robotic perturbations). Real-world physical deployment was tested on a Franka Research 3 robot under the DROID setup.
  • RLVR Post-Training: Evaluated using Qwen3-1.7B and Qwen3-4B models on the MATH (levels 3–5, tested on MATH500) and GSM8K datasets using GRPO and GMPO reinforcement learning algorithms.

Key Insights and Ideas

  • Structural Rank Heterogeneity: In VLA models, the vision and language modules process high-entropy data, whereas the action head outputs low-dimensional control vectors (e.g., 7D end-effector movements). Uniformly whitening the action head over-amplifies non-generalizable noise directions.
  • RLVR is Inherently Noisier than SFT: Policy gradients in RLVR have significantly lower SNR than Supervised Fine-Tuning (SFT) due to sparse trajectory-level rewards and stabilization mechanisms like clipping or importance sampling.
  • Preserving Pretrained Heterogeneity: Attention layers exhibit highly specialized per-head weight norms after pretraining. Treating a layer as a single block (as Muon does) erases this structure, whereas Pion's per-head mode preserves it.

Key Results

  • VLA Success Rates: On the LIBERO Object benchmark, Pion achieved a 100% success rate after just 1,500 training steps using a VLA-Adapter, compared to 97.0% for Muon and a dismal 32.2% for AdamW. On the real Franka robot, Pion achieved an average success rate of 85.6% across three grasp-and-place tasks, vastly outperforming Muon (38.9%) and AdamW (31.1%).
  • RLVR Stability: While Muon completely collapsed to 0% accuracy on MATH and GSM8K benchmarks due to noise amplification, Pion successfully stabilized training and consistently outperformed AdamW across both GRPO and GMPO algorithms.
Validation accuracy curves showing Muon's collapse and Pion's superior performance in RLVR.

Limitations and Future Work

Pion is explicitly tailored for regimes where informative gradients concentrate in a few leading singular values. Because it filters out the tail, it is expected to underperform Muon in standard LLM pretraining, where uniform whitening and maximum spectral exploration are highly beneficial.

The authors note that future work should explore how to dynamically adapt the high-pass cutoff to seamlessly recover Muon's exploratory behavior during pretraining while retaining Pion's robust noise resilience during post-training and cross-modality adaptation.

Summary updated

Abstract

Muon is a matrix-aware optimizer that leverages Newton-Schulz (NS) iterations to enforce spectral gradient orthogonalization by driving all singular values of the momentum matrix toward 1. While this uniform spectral whitening enhances exploration and outperforms AdamW in LLM pretraining, we show it could lead to fundamental limitations beyond pretraining in two regimes: (i) cross-modality vision-language-action (VLA) training, where inherently low-rank action-module gradients cause amplification of noisy tail directions, and (ii) reinforcement learning with verifiable rewards (RLVR), where low-SNR gradients and the need to preserve per-head specialization from prior training make whitening unstable. To address these challenges, we propose Pion, a drop-in replacement for Muon that preserves its computational efficiency while replacing uniform spectral whitening with a two-stage Promotion+Suppression mechanism, which we call the high-pass NS iteration. This design induces a sharp spectral high-pass effect, anchoring dominant singular values at 1 while suppressing noisy tail components toward 0, with controllable filter strength. To preserve pretrained per-head heterogeneity, Pion also supports a per-head mode that applies updates independently across attention heads via a simple reshape, at no extra cost. In VLA training on LIBERO and LIBERO-Plus, Pion consistently outperforms both baselines across l_1-regression (VLA-Adapter) and flow-matching (VLANeXt) architectures, e.g., reaching 100% success rate on LIBERO Object after 1,500 training steps with VLA-Adapter, vs. 97.0% for Muon and only 32.2% for AdamW. The advantage of Pion further extends to a real Franka Research 3 robot with a pi_0.5 backbone under the DROID setup on three grasp-and-place tasks. In RLVR post-training on Qwen3-1.7B/4B with GRPO and GMPO, Pion also outperforms AdamW on MATH and GSM8K while Muon collapses to zero.
BibTeX
@misc{fan2026rethinking,
  author = {Chongyu Fan and Gaowen Liu and Mingyi Hong and Ramana Rao Kompella and Sijia Liu},
  title = {{Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR}},
  year = {2026},
  eprint = {2605.19282},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.19282},
}

Loading related content...

Figure