LoRN: Low-Rank Noise Steering for Generative Robot Policies

Wei Liu, Abrar Anwar, Huihua Zhao, Soha Pouya, Yan ChangarXiv 2026 · arXiv

TL;DR

LoRN (Low-Rank Noise Steering) is a new reinforcement learning method that improves large, pretrained generative robot policies without updating any of their billions of weights. By steering a small, low-rank subspace of the initial noise—specifically targeting video noise in world-action models—LoRN achieves massive performance gains and enables seamless sim-to-real transfer.

What Problem Does This Paper Address?

Modern robot "foundation models," such as vision-language-action models (VLAs) and world-action models (WAMs), are incredibly capable but notoriously difficult to adapt to specific tasks. Traditional fine-tuning is computationally expensive and risks "forgetting" general capabilities.

A promising alternative is noise steering: instead of changing the model, you optimize the initial noise sample that the model "denoises" into an action. However, existing steering methods hit a wall with large models for two reasons:

  1. Dimensionality: Large models use high-dimensional noise (e.g., over 500,000 dimensions for video noise), creating a search space too vast for standard reinforcement learning.
  2. Norm Sensitivity: These models are trained on Gaussian noise that lives on a very specific "thin shell" (a high-dimensional sphere). If steering pushes the noise norm even slightly outside this narrow band, the model's performance collapses.
LoRN enables low-rank noise steering of large generative robot policies by selecting a noise direction within a fixed low-rank subspace.

Methods

The authors introduce Low-Rank Noise Steering (LoRN) to overcome these hurdles. The core idea is to separate the control dimension from the full noise dimension.

  • Low-Rank Projection: A small RL actor outputs a low-dimensional vector (e.g., just 32 coefficients). This vector is mapped to the full high-dimensional noise space using a fixed, random orthonormal projection matrix. This keeps the learning problem manageable regardless of the model's size.
  • Norm-Controlled Construction: To keep the model happy, LoRN carefully mixes the actor's steered direction with a fresh Gaussian sample. This mathematical construction ensures the resulting noise preserves the expected squared norm, keeping it firmly within the model's tolerated "shell."
  • Video Noise Steering: Uniquely, the authors apply this to the video noise of world-action models. Since these models jointly predict future video and actions, steering the video noise provides a surprisingly expressive "lever" for shaping the resulting actions.

Datasets

The researchers evaluated LoRN across a diverse set of benchmarks and models:

  • World-Action Models: Cosmos3-Edge, DreamZero-14B, and DreamZero-5B.
  • Action Policies: $\pi_0$ (a flow-matching VLA) and standard Diffusion Policies.
  • Simulation Environments: RoboLab (for generalist tasks), LIBERO (for long-horizon manipulation), and ALOHA (for bimanual tasks).
  • Physical Hardware: A bimanual robot station (YAM) was used to test sim-to-real transfer on tasks like "cube to bowl" and "marker to cup."

Key Insights and Ideas

  • Video Noise is the Secret Lever: One of the paper's most striking findings is that world-action models are far more sensitive to video noise than action noise. Steering the video noise resulted in significantly larger performance gains, suggesting it's a more expressive input for behavior modification.
  • The "Thin Shell" is Fragile: The authors quantified just how sensitive large models are to noise scale. For some models, a deviation of just 5% from the typical Gaussian norm causes success rates to drop to zero. This explains why previous steering methods often failed on large-scale models.
Figure 2: Frozen policies tolerate only a band of noise norms. Success when Gaussian samples are scaled by 𝑐, relative to 𝑐= 1. The shaded region marks the narrow band of Cosmos3-Edge.
  • Random Subspaces are Sufficient: You don't need a complex method to find "good" noise directions. The paper proves that a random low-rank subspace is highly likely to contain directions that improve the base policy, making simple random projections a powerful tool.

Key Results

  • Massive Success Boost: Across ten model-task pairs for world-action models, video steering with LoRN improved average success by 26.2 percentage points over the base policy. For comparison, the previous state-of-the-art (DSRL) only managed a 7.1-point gain.
  • Sim-to-Real Success: Steering actors trained entirely in simulation transferred directly to physical hardware. On a bimanual robot, LoRN enabled successful task completion (e.g., 6/10 for "cube to bowl") where the base policy failed every single trial.
  • Sample Efficiency: Because it operates in a compact 32-dimensional space, LoRN often reaches its peak performance much faster than full-dimensional steering methods.

Limitations and Future Work

While powerful, LoRN has its limits. It relies on behaviors that the frozen policy is already capable of producing but might not trigger with default sampling. It cannot "teach" a model a brand-new skill it hasn't seen during pretraining.

The authors also note that the choice of rank and steering strength remains empirical. Future work could explore more adaptive ways to choose these parameters or investigate if steering can be combined with sparse weight updates for even greater flexibility.

Summary updated

Abstract

Noise steering improves pretrained generative robot policies by learning to select their initial noise without updating model weights. We find that actions in the evaluated world-action models are more sensitive to video noise than action noise, making video noise an expressive input for steering. However, the noise-steering methods we evaluate, in their published forms, struggle to scale to the high-dimensional noise inputs of large policy classes such as world-action models. We introduce Low-Rank Noise Steering (LoRN), a reinforcement learning method that scales noise steering to world-action models. A small steering actor selects a direction through a fixed low-rank random projection, while our norm-controlled construction keeps the resulting noise near the typical scale of a Gaussian sample. This separates the actor's control dimension from the full noise dimension, enabling steering of both video and action inputs. Across three world-action models and ten model-task pairs, video steering improves average success by 26.2 percentage points over the base policy, compared with 7.1 points for DSRL action steering. For smaller action policies, the same construction achieves comparable final success while generally improving early-training success. Steering actors trained in simulation also transfer to three physical bimanual manipulation tasks across two base policies, without additional real-world training.
BibTeX
@misc{liu2026lorn,
  author = {Wei Liu and Abrar Anwar and Huihua Zhao and Soha Pouya and Yan Chang},
  title = {{LoRN: Low-Rank Noise Steering for Generative Robot Policies}},
  year = {2026},
  eprint = {2611.23628},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23628},
}

Loading related content...

Figure