Topology-Aware 4D Gaussian Splatting for Egocentric Videos

Leonardo Vanni, Stefano Gasperini, Federico Tombari, Chiara PlizzariarXiv 2026 · arXiv

Reconstructing the world from a first-person perspective is a cornerstone of embodied AI and augmented reality. While 4D Gaussian Splatting (4DGS) has recently revolutionized dynamic 3D reconstruction, it harbors a fundamental flaw: it assumes the "topology" of the world is fixed. In other words, it expects objects to deform but never to split, merge, or reveal hidden interiors.

In a new paper, researchers from Bocconi University, TUM, and Google introduce Topo-4DGS, a framework designed to handle the messy, topology-changing reality of human-object interactions.

TL;DR

Topo-4DGS is a topology-aware reconstruction method that enables 4D Gaussian Splatting to handle events like cutting, assembling, or opening objects in egocentric videos. By modeling independent part-level motions and dynamically activating Gaussians as new surfaces are exposed, it significantly reduces visual artifacts and outperforms state-of-the-art methods on a newly introduced benchmark, TopoEgo.

What Problem Does This Paper Address?

Most current 4DGS methods represent dynamic scenes by deforming a fixed set of 3D Gaussian primitives from a "canonical" space to each video frame. This works well for a person waving their arms, but it fails catastrophically during everyday tasks like slicing a potato or opening a lunchbox.

When an object is cut, new surfaces are created. When a box is opened, a previously concealed interior is revealed. Because standard 4DGS relies on a fixed representation, it tries to "stretch" existing Gaussians to cover these new areas, resulting in severe "ghosting" artifacts or missing geometry. The authors found that these topological changes occur in over 68% of Ego4D and 80% of EPIC-KITCHENS videos, making this a critical hurdle for egocentric vision.

Comparison of Topo-4DGS against baselines on a potato-slicing task. Standard methods produce "ghosting" or fail to capture the multiple slices.

Methods

The authors propose two primary mechanisms to make 4DGS topology-aware:

  1. Motion Bifurcation: Instead of a single deformation field, Topo-4DGS uses an "event tree" to define how an object splits into independent parts. A Spacetime Clustering Field learns to route Gaussians to specific "motion branches." Before a cut, parts move together rigidly; after the cut, they follow independent trajectories.
  2. Topological Boundary Resolution: To handle appearing and disappearing surfaces, the method dynamically manages Gaussian "support":
  3. Boundary Construction: When two pieces separate, the system identifies the contact area and spawns new Gaussians to fill the newly exposed interior faces.
  4. Event-Conditioned Activation: Each Gaussian is assigned a temporal activation window. Gaussians representing the interior of a closed box are deactivated until the "open" event occurs.
Overview of the Topo-4DGS pipeline, showing how Motion Bifurcation and Boundary Resolution work together at the event time.

Datasets

The researchers introduced TopoEgo, the first benchmark specifically dedicated to topological changes in egocentric video.

  • Source: Curated from the massive EPIC-KITCHENS and Ego4D datasets.
  • Composition: 24 real-world sequences featuring 34 topological events across four categories: Fission (cutting/breaking), Fusion (assembling), Open, and Close.
  • Annotations: Includes event times, event trees (how parts relate), camera poses, and hand-object masks.
Examples of the four event families in the TopoEgo benchmark: fission, fusion, open, and close.

Key Insights and Ideas

  • Branching vs. Deforming: The core "aha" moment is that topological changes are discrete structural events, not continuous deformations. By explicitly branching the motion model at specific timestamps, the optimization remains stable.
  • Soft Routing for Hard Splits: While the physical split might be sudden, the "Spacetime Clustering Field" uses a soft routing mechanism (via a smooth event gate) that allows the model to gracefully transition Gaussians between motion branches during the event window.
  • The "Inside" Matters: Simply allowing parts to move independently isn't enough; you must account for the "exposed interior." The automated spawning of Gaussians at the contact boundary is what prevents the "hollow" look common in other reconstruction methods.

Key Results

Topo-4DGS sets a new state-of-the-art across the board on the TopoEgo benchmark:

  • Quantitative: It achieves a massive +4.47 dB PSNR improvement on post-event frames compared to the strongest baseline. This is where fixed-topology assumptions fail most severely.
  • Qualitative: As seen in Figure 4, Topo-4DGS is the only method that maintains clean geometry for independent slices of a pizza or the interior of a container after it is opened.
  • Robustness: The method preserves performance on standard rigid objects, ensuring that the topology-aware features don't introduce "quality trade-offs" in simpler scenes.
Qualitative comparison showing Topo-4DGS successfully reconstructing fission, fusion, open, and close events where baselines fail.

Limitations and Future Work

While a major step forward, the authors note several areas for improvement:

  • Manual Specification: The main results rely on manually provided event times and piece counts. While they demonstrate an "Automatic Topology Discovery" module using Vision-Language Models (VLMs) in the appendix, integrating this into an end-to-end differentiable pipeline is a future goal.
  • Piecewise Rigidity: The model assumes parts are approximately rigid. It struggles with "strongly deformable" objects like bread dough or soft clay that stretch significantly before breaking.
  • Occlusion: Like all monocular methods, it is limited by what the camera can see. If a surface is never exposed to the camera, it cannot be reconstructed accurately.

Summary updated

Abstract

Reconstructing dynamic scenes and objects from monocular egocentric video is crucial for embodied AI, yet existing 4D Gaussian Splatting (4DGS) approaches struggle with everyday interactions involving topological changes, such as cutting, assembly, opening, or closing. Since 4DGS methods rely on deforming a fixed representation, events that split, merge, or reveal surfaces cause severe visual artifacts. To address this challenge, we introduce TopoEgo, the first benchmark dedicated to topological changes in egocentric video (covering fission, fusion, open, and close), alongside Topo-4DGS, a topology-aware reconstruction method. Topo4DGS jointly models part-level motion and time-varying surface support by assigning independent rigid transformations to interacting parts and dynamically activating or deactivating Gaussians as surfaces appear, disappear, or separate. Extensive experiments demonstrate that Topo-4DGS significantly outperforms state-of-the-art 4DGS approaches across topological changes, especially on post-event frames. Code and data will be made publicly available.
BibTeX
@misc{vanni2026topology,
  author = {Leonardo Vanni and Stefano Gasperini and Federico Tombari and Chiara Plizzari},
  title = {{Topology-Aware 4D Gaussian Splatting for Egocentric Videos}},
  year = {2026},
  eprint = {2611.16012},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16012},
}
Figure