MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation

Rong Shan, Aofan Yu, Bo Chen, Kuo Cai, Qiang Luo, Ruiming Tang, Han Li, Weiwen Liu, Weinan Zhang, Jianghao LinarXiv 2026 · arXiv

TL;DR

MuonRec introduces the recently proposed Muon optimizer to recommender systems, breaking the industry's default reliance on Adam/AdamW. By applying orthogonalized momentum updates to 2D weight matrices, MuonRec accelerates training convergence by an average of 32.4% while simultaneously boosting recommendation quality (NDCG@10) by 12.6%.

What Problem Does This Paper Address?

Recommender systems (RecSys) are undergoing a massive scaling revolution, with modern architectures expanding to over a billion parameters to handle web-scale interaction data. However, while model architectures have evolved rapidly, the choice of optimizer has remained stagnant: almost every modern RecSys pipeline defaults to Adam or AdamW.

The authors argue that this unexamined default is sub-optimal. In deep Transformer-based recommenders, coordinate-wise adaptive optimizers like Adam can bias weight updates toward a low-rank subspace, leading to "representation collapse." This paper addresses this gap by investigating whether matrix-structured optimization can overcome these limitations and provide more efficient, high-quality training for large-scale recommendation models.

Methods

The authors introduce MuonRec, a framework that adapts the Muon optimizer for recommendation tasks using a hybrid optimization strategy. Parameters are partitioned into two distinct groups:

  • The Muon Group: Applied to all 2D weight matrices inside Transformer and sequence modeling blocks (such as attention projections and Multi-Layer Perceptron weights). Instead of updating weights coordinate-by-coordinate, Muon calculates gradient momentum and applies Newton-Schulz iteration to approximately orthogonalize it. This treats the parameters as linear operators, promoting diverse update directions and preventing the model from converging along only a few dominant eigenvectors.
  • The Adam Group: Applied to 1D parameters (biases, LayerNorm), embeddings, and output layers (such as semantic ID embeddings and softmax heads), where matrix-structured orthogonalization does not naturally apply.

Datasets

The framework was evaluated on two public datasets from the Amazon Review collection:

  1. Industrial and Scientific: Comprising 1,987 users, 3,686 items, and 13,555 interactions.
  2. Office Products: Comprising 2,024 users, 3,459 items, and 13,929 interactions.

Both datasets were preprocessed using standard 5-core filtering and a chronological leave-one-out strategy. The authors tested MuonRec across traditional sequential recommenders (GRU4Rec, Caser, SASRec) and modern generative recommenders (TIGER and MiniOneRec, ranging from 0.5B to 3.0B parameters).

Key Insights and Ideas

  • Implicit Spectral Regularization: Muon acts as an implicit spectral regularizer. By forcing updates to be closer to orthogonal, it encourages a more uniform singular-value spectrum. This is particularly beneficial for generative recommenders, as it reduces "index collapse" during tokenization.
  • Hyperparameter Scale Invariance: A major headache in scaling large models is re-tuning hyperparameters. The authors discovered that Muon's optimal learning rate ($\eta \approx 10^{-2}$) remains completely stable across model scales from 0.5B to 3.0B parameters. This allows engineers to safely tune hyperparameters on small proxy models before deploying to billion-scale giants.
  • Multi-Stage Trajectory Consistency: For generative models that undergo both Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), using Muon in both stages yields the best results. Introducing Muon only during the RL phase can disrupt optimization momentum because the base model was originally warmed up using Adam's dynamics.

Key Results

MuonRec consistently outperformed Adam/AdamW baselines in both speed and accuracy:

  • Faster Convergence: MuonRec reduced converged training steps by an average of 32.4% across all settings. For instance, the generative model TIGER converged roughly 3× faster when using Muon on the Industrial dataset.

    Figure 1: Training dynamics of TIGER [27] on Industrial (left) and Office (right) datasets. The top row displays train- ing loss, while the bottom row reports the NDCG@10 curves. The Muon optimizer demonstrates significantly faster con- vergence and superior final performance compared to Adam.
  • Better Recommendation Quality: Beyond speed, final ranking metrics saw substantial improvements. Traditional models achieved an average boost of 15.1% in Recall and 14.5% in NDCG. Generative models saw even larger jumps; TIGER achieved a massive +54.2% Recall and +56.0% NDCG on the Office dataset.

  • Robust Scaling: On the 1.5B parameter MiniOneRec model, Muon cut training steps by over 25% while maintaining superior ranking performance over Adam.
Figure 2: Impact of LR on NDCG@10 and training loss.

Limitations and Future Work

The authors noted a performance drop when scaling MiniOneRec from 1.5B to 3.0B parameters under both optimizers. They attribute this to overfitting caused by the limited size of the evaluation datasets.

Future work will focus on testing MuonRec against much larger, web-scale datasets to unlock the full potential of the 3B+ parameter models. Additionally, the authors suggest exploring further hardware-level optimizations to reduce the computational overhead of the Newton-Schulz iterations during training.

Summary updated

Abstract

Recommender systems (RecSys) are increasingly emphasizing scaling, leveraging larger architectures and more interaction data to improve personalization. Yet, despite the optimizer's pivotal role in training, modern RecSys pipelines almost universally default to Adam/AdamW, with limited scrutiny of whether these choices are truly optimal for recommendation. In this work, we revisit optimizer design for scalable recommendation and introduce MuonRec, the first framework that brings the recently proposed Muon optimizer to RecSys training. Muon performs orthogonalized momentum updates for 2D weight matrices via Newton-Schulz iteration, promoting diverse update directions and improving optimization efficiency. We develop an open-source training recipe for recommendation models and evaluate it across both traditional sequential recommenders and modern generative recommenders. Extensive experiments demonstrate that MuonRec reduces converged training steps by an average of 32.4\% while simultaneously improving final ranking quality. Specifically, MuonRec yields consistent relative gains in NDCG@10, averaging 12.6\% across all settings, with particularly pronounced improvements in generative recommendation models. These results consistently outperform strong Adam/AdamW baselines, positioning Muon as a promising new optimizer standard for RecSys training. Our code is available.
BibTeX
@misc{shan2026muonrec,
  author = {Rong Shan and Aofan Yu and Bo Chen and Kuo Cai and Qiang Luo and Ruiming Tang and Han Li and Weiwen Liu and Weinan Zhang and Jianghao Lin},
  title = {{MuonRec: Shifting the Optimizer Paradigm Beyond Adam in Scalable Generative Recommendation}},
  year = {2026},
  eprint = {2603.00416},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.00416},
}
Figure