LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild

Linkai Liu, Yuntian Zhang, Zhenshan Bing, Chen Chen, Lingjuan Lyu, Shangguang Wang, Mengwei Xu, Dongqi CaiarXiv 2026 · arXiv

TL;DR

LiteNWM is a lightweight "latent world model" that gives robots the foresight to choose better paths without the massive computational cost of traditional models. By predicting future states in a compressed latent space rather than generating full RGB images, it achieves a staggering 128x speedup over state-of-the-art baselines, enabling real-time, future-aware navigation on physical robots.

What Problem Does This Paper Address?

Modern visual navigation policies (like NoMaD or ViNT) are great at proposing potential paths for a robot to follow. However, they often struggle in complex environments because they don't explicitly "think ahead" about the consequences of their actions.

Previous attempts to solve this used Navigation World Models (NWMs) to perform "visual rollouts"—essentially imagining what the robot would see if it took a specific path. While effective, these models are incredibly slow. Generating high-quality RGB images for dozens of candidate trajectories is a massive bottleneck, often taking hundreds of seconds per decision. This makes them impractical for real-time deployment on mobile robots with limited onboard hardware.

The latency bottleneck: RGB rollouts (CDiT) dominate the computation time in existing NWMs.

Methods

LiteNWM tackles the efficiency problem by restructuring how a world model evaluates trajectories. Key innovations include:

  1. Latent-Space Rollouts: Instead of reconstructing full RGB images, LiteNWM predicts future features in a compressed latent space (using a DINOv3 backbone). This bypasses the expensive decoding step entirely.
  2. Shared Visual Encoding: The model encodes the visual history and goal once per decision state, sharing these features across all candidate trajectories.
  3. Joint Multi-Horizon Prediction: Rather than predicting one step at a time (autoregressively), LiteNWM predicts multiple future horizons (H1–H4) simultaneously.
  4. Incumbent-Relative Scorer: A learned scoring module compares new trajectory proposals against the current "incumbent" path. It uses the predicted latent futures to rank candidates based on how well they align with the goal and avoid errors.
  5. Pose-Conditioned Warping: To give the model a head start, it uses the robot's intended motion to "warp" current visual features into their expected future positions before refining them with a transformer-based predictor.

Datasets

The authors evaluated LiteNWM across three diverse navigation datasets:

  • RECON: Diverse open-world navigation trajectories.
  • SCAND: Social navigation data spanning indoor and outdoor environments.
  • SACSoN: A dataset focused on complex indoor robot-pedestrian interactions.

Training involved a large-scale offline phase followed by real-world validation on a Unitree Go2 EDU quadruped robot.

Key Insights and Ideas

  • RGB is Overkill for Ranking: The "aha moment" is that you don't need to see a pretty picture to know if a path is good. Latent features contain enough semantic information to distinguish between a successful trajectory and one that leads to a collision or a dead end.
  • The Bottleneck is Generative: In NWM-XL, candidate-conditioned RGB rollouts account for 97.6% of the total computation time. By eliminating this, LiteNWM moves the needle from "offline research tool" to "onboard real-time system."
  • Zero-Shot Transfer: The LiteNWM evaluator is modular. It can be paired with different navigation policies (like NoMaD or MBRA) without needing to be retrained for each one, proving it learns general principles of "good" vs. "bad" future states.

Key Results

LiteNWM sets a new bar for efficient, future-aware navigation:

  • Speed: Achieves a 128.00× end-to-end speedup over NoMaD+NWM-XL on an RTX 5090. On a robot's onboard Jetson Orin, it makes decisions in ~2.2 seconds compared to the ~235 seconds required by previous NWM variants.
  • Accuracy: Reduces macro-averaged trajectory error by 17.56% relative to the strongest baseline.
  • Real-World Success: In physical robot experiments, LiteNWM improved the navigation success rate from 43.3% to 83.3% compared to a standard NoMaD policy.
Fig. 1. Overview of LiteNWM and main results. (a) LiteNWM predicts and scores action-conditioned future trajectories from the visual history and goal; (b) Closed-loop real-world/simulation execution of the trajectory selected. SR: success rate.; (c) Accuracy–efficiency comparison.

Limitations and Future Work

While LiteNWM is a major step forward, the authors note a few areas for improvement:

  • Short Horizon: The current model only looks 4 steps ahead. While this is enough for local planning, longer-range foresight might be needed for more complex global reasoning.
  • Frozen Proposer: Currently, the policy that proposes paths is "frozen." The authors aim to eventually train the trajectory generator and the evaluator together so the generator can learn directly from the world model's feedback.
  • Beyond Navigation: The team plans to extend this action-conditioned latent foresight to more complex tasks, such as robotic manipulation.

Summary updated

Abstract

Direct visual navigation policies generate trajectories efficiently but do not explicitly evaluate their future consequences. Generative navigation world models provide this foresight through visual rollouts, which are costly when evaluating multiple candidates. We present LiteNWM, a latent navigation world model that shares visual encoding across candidates and jointly predicts their action-conditioned future representations at multiple horizons, while a learned scorer uses these predictions to select trajectories. In offline evaluations on RECON, SCAND, and SACSoN, LiteNWM reduces macro-averaged trajectory error by 17.56% relative to NoMaD+NWM-XL and achieves a 128.00-fold end-to-end speedup on an RTX 5090. The same evaluator transfers from NoMaD to MBRA without proposer-specific retraining, reducing MBRA's macro-averaged trajectory error by 16.2%. In real-robot experiments in unseen indoor and outdoor environments, LiteNWM improves navigation success from 43.3% to 83.3% relative to NoMaD. These results demonstrate that LiteNWM can be deployed for future-aware planning and closed-loop navigation on a physical robot.
BibTeX
@misc{liu2026litenwm,
  author = {Linkai Liu and Yuntian Zhang and Zhenshan Bing and Chen Chen and Lingjuan Lyu and Shangguang Wang and Mengwei Xu and Dongqi Cai},
  title = {{LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild}},
  year = {2026},
  eprint = {2610.12368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.12368},
}
Figure