GeoState: Learning View-Decoupled Geometry States for 3D World Modeling

Hongchang Liu, Jian Liu, Yujuan Ding, Liang Peng, Xun Jiang, Fei Ma, Xing Xu, Yi BinarXiv 2026 · arXiv

TL;DR

GeoState introduces a novel 3D world modeling approach that decouples scene geometry from the camera's viewpoint. By learning a "view-decoupled" state representation, the model can predict future scene evolution more accurately over long horizons and support consistent depth querying from any arbitrary camera angle without needing per-frame scale alignment.

What Problem Does This Paper Address?

Predicting how a 3D environment evolves is a cornerstone of autonomous driving and robotics. While recent "world models" have moved from predicting raw pixels to using 3D representations, they often rely on features from Geometry Foundation Models (GFMs) like VGGT.

The catch? These GFM features are typically view-dependent. This means the internal representation of a scene changes simply because the camera moves, even if the physical world remains static. This "view-coupling" entangles camera motion with actual scene dynamics, making the job of a predictive model much harder. Imagine trying to predict the path of a ball while someone is shaking the camera; it's far easier if you can first stabilize the footage. GeoState aims to provide that "stabilized" 3D latent space.

Methods

The authors propose a framework divided into two main components: GeoState Learning and GeoState World Modeling.

1. GeoState Learning

As shown in the overview (Figure 1), the system first learns to compress multi-view observations into a compact, view-independent state:

  • Scene Tokenizer: This module takes geometry-aware features from a frozen GFM (VGGT-Ω) and aggregates them into a fixed-length "GeoState" represented in a shared coordinate system.
  • Geometry Decoder: To ensure the GeoState actually captures the 3D structure, a decoder queries this state using camera rays to reconstruct depth maps. Crucially, the query viewpoint is introduced only at decoding time, meaning the same state can be queried from any angle.
Figure 1: Overview of GeoState. GeoState Learning constructs view-decoupled scene states with a scene tokenizer and query-conditioned geometry decoder. GeoState World Modeling predicts future states directly in this representation space.

2. GeoState World Modeling

Once the view-decoupled representation is learned, the authors train a Geometry World Model:

  • It uses a masked factorized Transformer to predict future GeoStates autoregressively.
  • Because the state is decoupled from the view, the model can focus purely on physical scene dynamics (e.g., a car moving forward) while camera motion is handled by a separate, optional predictor.
  • Residual Updates: The model predicts the change ($\Delta S$) between states, which naturally encourages scene persistence.

Datasets

The model was trained and evaluated across several high-profile autonomous driving datasets:

  • Pretraining: A mix of large-scale 3D datasets including DL3DV, TartanAir, RealEstate10K, CO3D, and WildRGB-D.
  • Main Evaluation: The Waymo Open Dataset, specifically partitioned into "Static" and "Dynamic" windows to test how well the model captures movement.
  • Zero-Shot Generalization: The model was tested on KITTI and nuScenes without any fine-tuning on those specific datasets to prove its robustness.

Key Insights and Ideas

  • The "Aha" Moment - View Decoupling: The authors demonstrate that a static scene undergoes massive changes in feature space if the camera moves. By forcing the representation to be view-independent, the temporal dynamics become significantly simpler and more predictable.
  • Scale Consistency: A major headache in 3D forecasting is "scale drift," where the size of the world seems to change over time. GeoState calibrates the metric scale once at the start of a sequence and maintains it throughout the rollout, eliminating the need for the per-frame scale alignment required by prior methods.
  • GFM Context Sensitivity: The researchers discovered that GFM features are sensitive to the entire context of a forward pass. GeoState’s tokenizer helps suppress this "context noise," focusing only on the underlying geometry.

Key Results

GeoState sets a new state-of-the-art for 3D geometry forecasting:

  • Superior Accuracy: On the Waymo dataset, GeoState consistently outperformed prior models like VGGT-World and FR3D. At a 2.0-second forecasting horizon, it achieved a 40.4% reduction in Absolute Relative (AbsRel) error compared to VGGT-World.
  • Geometric Consistency: As shown in Figure 3, the model maintains high consistency when queried from different virtual viewpoints, proving the state is truly 3D-aware rather than just a clever 2D projection.
  • Zero-Shot Prowess: The model outperformed existing baselines on KITTI and nuScenes even without seeing a single training frame from those datasets, highlighting the transferability of the learned GeoState.
Figure 2: Qualitative multi-view depth forecasting. Red boxes highlight prediction differences.

Limitations and Future Work

  • Depth-Only Focus: Currently, GeoState focuses on geometry (depth). The authors note that future iterations could incorporate semantic information (what objects are) and appearance (RGB textures) into the decoupled state.
  • Frozen Backbones: The model relies on a frozen GFM. While this makes training more efficient, end-to-end fine-tuning of the entire pipeline might unlock even higher performance at the cost of significant computational resources.
  • Embodied AI: The authors suggest this framework is a natural fit for embodied agents, where a stable, predictable world model is essential for safe planning and interaction.

Summary updated

Abstract

Recent 3D world models increasingly use features from geometry foundation models (GFMs) as predictive world states, benefiting from their rich geometric priors. However, the view dependence of these features entangles scene dynamics with camera-induced variation, making the resulting state evolution harder to predict. We propose GeoState, a view-decoupled geometry representation for predictive 3D world modeling that separates scene-state evolution from camera-dependent observation. Specifically, a scene tokenizer aggregates multi-view GFM features into a view-decoupled scene state, which a geometry decoder queries using camera rays. An autoregressive world model then predicts future scene states, while camera motion is modeled separately. The predicted states can therefore be decoded under either predicted or externally specified cameras, while retaining a single context-derived scale throughout the rollout without per-frame scale alignment. Experiments show that GeoState outperforms prior 3D world models on Waymo, supports geometrically consistent cross-view querying, and generalizes zero-shot to KITTI and nuScenes. Project page: https://hongchang-liu.github.io/GeoState-project-page/
BibTeX
@misc{liu2026geostate,
  author = {Hongchang Liu and Jian Liu and Yujuan Ding and Liang Peng and Xun Jiang and Fei Ma and Xing Xu and Yi Bin},
  title = {{GeoState: Learning View-Decoupled Geometry States for 3D World Modeling}},
  year = {2026},
  eprint = {2610.22343},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.22343},
}
Figure