Disentangling Static Scenes from Transient Occluders in the Wild

This direction is split into other directions: transient occluder interference. Its last version stays readable and citable here.

Version 1 of 161 papers76 conceptsfrozen 2026-10-04

When we try to reconstruct a 3D model of a landmark like the Trevi Fountain using internet photos, we encounter a fundamental conflict. Standard 3D reconstruction algorithms assume the world is frozen in time. However, a collection of Flickr photos is anything but static: tourists walk through the frame, cars park and leave, and the lighting shifts from high noon to twilight.

In research, these temporary elements are called transient objects. If a model tries to find a single consistent 3D representation for all pixels across these images, it fails. It attempts to "average" the tourists into the stone, resulting in "ghosting" artifacts—blurry, semi-transparent clouds—and "floaters" that corrupt the permanent geometry of the scene.

The core intuition of this research direction is disentanglement: explicitly separating the permanent, static scene from the temporary, transient occluders.

The Mechanism of Disentanglement

The breakthrough in handling "in-the-wild" data came from moving away from a single model for the entire image. Instead, researchers like Ricardo Martin Brualla and Mehdi S. M. Sajjadi in NeRF in the Wild proposed splitting the neural representation into two distinct branches:

  1. The Static Branch: This captures the permanent geometry and appearance (the buildings, the statues). It is shared across all images in the collection.
  2. The Transient Branch: This captures the "noise"—the pedestrians and vehicles. This branch is per-image and models the uncertainty of each pixel.

By allowing the model to "blame" inconsistent pixels on the transient branch, the static branch is freed to learn a clean, high-fidelity reconstruction of the underlying landmark. This process effectively "erases" tourists by identifying them as outliers that do not follow consistent geometric projections across multiple viewpoints.

Identifying What to Ignore

How does a model know which pixels are "permanent" and which are "transient" without manual labels? The field uses several clever unsupervised strategies:

  • Visibility and Uncertainty Maps: Methods like GS-W by Dongbin Zhang use 2D visibility maps to ignore transient occluders during optimization. If a pixel's color cannot be explained by the 3D model, the system assigns it a high uncertainty score and ignores it.
  • Semantic Priors: WildGaussians by Jonas Kulhanek and Songyou Peng integrates DINO-based features to predict which parts of an image are likely to be transient, allowing the model to proactively filter out crowds.
  • Multi-View Consistency: MVGSR by Chenfeng Hou leverages the fact that static objects appear in the same relative 3D space across views, while "distractors" like moving pedestrians do not. By pruning elements that lack multi-view feature consistency, the system produces clean 3D meshes.

Handling Appearance Variations

Transient objects aren't the only challenge; "in-the-wild" photos also suffer from inconsistent lighting and weather. To solve this, researchers use appearance embeddings.

As shown in SWAG and WE-GS, each image is assigned a unique "latent code" that describes its specific lighting conditions (e.g., "sunny afternoon" vs. "rainy evening"). This allows the model to separate the intrinsic color of the stone from the extrinsic light hitting it. WildCAT3D by Morris Alper takes this further, enabling "appearance transfer" where a scene reconstructed from day photos can be re-rendered to look like a night shot.

The Shift to Real-Time Reconstruction

While early methods like Neural Rerendering in the Wild were slow, the field has recently pivoted toward 3D Gaussian Splatting (3DGS).

Techniques like Wild-GS by Jiacong Xu and Vishal M. Patel adapt the speed of 3DGS to unconstrained collections. By modeling the scene as a collection of 3D Gaussians rather than a continuous neural field, these models can train 200x faster and render 10,000x faster than previous NeRF-based solutions, making it possible to create navigable virtual tours of crowded landmarks in minutes.

Beyond Landmarks: Diverse Environments

This "static-transient" framing is now being applied to specialized domains where occlusions are even more aggressive:

  • Underwater: UW-GS treats fish and floating debris as transient distractors to recover clean seafloor geometry.
  • Autonomous Driving: Event-Based De-Snowing by Davide Scaramuzza identifies snowflakes as transient spatiotemporal "streaks" to see the road behind them.
  • Satellite Imaging: SatensoRF uses similar logic to filter out moving vehicles and clouds from multi-date satellite imagery.

By treating the world as a composition of permanent structures and temporary "noise," these researchers have turned cluttered, unorganized photo collections into a high-fidelity map of our physical world.

Papers in this version

Showing 60 of 61 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_disentangling_static_scenes_from_transie_d715832b,
  title = {Disentangling Static Scenes from Transient Occluders in the Wild},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-static-scenes-from-transient-occluders-in-the-wild/rdr_d715832b30f64c088db04b89998d4b00},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_disentangling_static_scenes_from_transie_d715832b_v1,
  title = {Disentangling Static Scenes from Transient Occluders in the Wild},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/disentangling-static-scenes-from-transient-occluders-in-the-wild/rdr_d715832b30f64c088db04b89998d4b00/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}