Higher Dimensional Deformation Fields for Topological Changes

17 concepts9 papers

When modeling a dynamic 3D scene—such as a video of a person moving, a shirt folding, or an object tearing—neural reconstruction methods face a fundamental mathematical bottleneck: the continuity paradox.

Most state-of-the-art methods represent dynamic scenes by learning a static "canonical" 3D shape and a neural deformation field (typically a Multi-Layer Perceptron, or MLP) that warps points from each video frame back to this canonical space. Because neural networks are continuous functions, the learned deformation field must be a homeomorphism—a continuous, bijective mapping.

This continuity is a double-edged sword. A continuous mapping cannot alter the topology of a shape. If a person opens their mouth, the lips must separate; if a piece of paper tears, a single surface must split into two. A standard 3D-to-3D deformation field cannot model these structural breaks. Forcing it to do so introduces severe geometric artifacts, such as unnatural stretching, "ghosting," or empty hollows where surfaces should cleanly separate.

To solve this, a powerful paradigm has emerged across recent research: lifting the canonical space into higher dimensions. By augmenting the standard 3D coordinate space with extra "ambient" or "hyper" dimensions, continuous neural networks can model complex topological discontinuities as smooth, continuous transformations.

The Intuition: Slicing Through Flatland

To understand why adding dimensions solves a topological problem, imagine a two-dimensional "Flatland" universe. If a 3D apple passes vertically through a Flatlander's 2D viewing plane, they will observe a small circle appear out of nowhere, grow larger, split into two separate shapes (the core and a leaf), and then shrink and vanish. To the Flatlander, this sequence represents a series of magical, discontinuous topological changes: geometry is appearing, tearing, and disappearing.

However, from our 3D perspective, the apple is a perfectly solid, continuous, static shape. The apparent topological breaks are purely an artifact of taking a 2D slice of a higher-dimensional object.

[ Higher-Dimensional Canonical Space (Smooth, Continuous Shape) ]
                             │
                             ▼  (Slicing / Projection)
[ 3D Physical Space (Apparent Tears, Splits, and Merges) ]

By mapping 3D physical coordinates into a higher-dimensional canonical space (e.g., 4D, 5D, or a latent hyperspace), we give the neural network an "escape hatch." The deformation network can execute smooth, continuous motions in the hyperspace, moving points around obstacles or separating them along the extra dimensions. When these higher-dimensional structures are sliced or projected back down to our 3D world, they manifest as clean topological breaks.

Topological Slicing in Radiance Fields

This higher-dimensional strategy was prominently brought to neural radiance fields by Keunhong Park et al. in their framework, HyperNeRF. Instead of restricting a deformation network to output a 3D canonical coordinate, they extend the input space to include additional "ambient" dimensions. They model topological variations by sweeping a deformable slicing plane through this higher-dimensional space, allowing the network to represent complex structural changes—like a mouth opening or a cup being lifted—without requiring discontinuous mathematical functions.

This concept can be extended beyond individual coordinate points to entire light rays. In DyLiN, Heng Yu et al. lift ray representations into a higher-dimensional hyperspace. This gives Light Field Networks (LFNs) the mathematical flexibility to handle non-rigid motion and objects breaking apart, while preserving the lightning-fast rendering speeds inherent to light field architectures.

For interactive applications, Chengwei Zheng et al. (EditableNeRF) apply this ambient mapping to sparse 3D keypoints. By transforming user-guided keypoints into a higher-dimensional space, their network can smoothly compute structural breaks during editing—such as a piano key being pressed down and separating from its resting plane—without introducing geometric distortion.

High-Dimensional Implicit Surfaces

The same hyper-dimensional insight applies to implicit surface reconstructions, such as Signed Distance Functions (SDFs), where maintaining strict surface integrity is vital.

3D Observation Point (x, y, z) + Time (t)
               │
               ▼ (Deformation Network)
Higher-Dimensional Canonical Space (x', y', z', w_1, w_2, ...)
               │
               ▼ (Implicit Function / SDF)
      Continuous Surface Geometry

When reconstructing dynamic scenes from multi-view or monocular videos, template-free methods often struggle when surfaces make or break contact. To address this, Decai Chen et al. developed DySurf, which maps points to a canonical space augmented with extra hyper-coordinates. These extra dimensions allow the neural network to represent fluid changes in connectivity that standard 3D warps cannot accommodate. Similarly, Hongrui Cai et al. (NDR) project 3D coordinates into a higher-dimensional latent space, enabling an invertible deformation network to maintain strict cycle consistency while smoothly handling splitting and merging surfaces.

To make these high-dimensional representations practical for real-time optimization, Lourdes Agapito and Mirgahney Mohamed (DynamicSurf) replace slow, pure-MLP architectures with an optimizable feature grid combined with a topology-aware coordinate lifting network. This allows the model to capture complex folds and appearing geometry with a massive 6x speedup in optimization time. Hengyi Wang et al. (MorpheuS) also leverage this hyper-dimensional mapping to solve the problem of unobserved regions in monocular tracking, mapping points from an observation frame into a static, higher-dimensional canonical space to complete complex 360-degree reconstructions.

Beyond Temporal Tracking: Category-Level Modeling

While lifting coordinates is highly effective for tracking an object over time, the mechanism is powerful enough to model topological variations across entirely different objects within a category.

For domain-specific tracking, such as human head modeling, Martin Rünz (MonoNPHM) demonstrates that augmenting a backward deformation field with additional dimensions allows a continuous parametric model to cleanly capture the opening and closing of a mouth. Because the model handles these discontinuities in the hyper-space, it completely bypasses the need for a fixed, pre-defined mesh template.

Taking this a step further, Shivam Duggal and Deepak Pathak (TARS) apply coordinate lifting to unsupervised single-view 3D reconstruction. Instead of tracking a single moving object, their framework learns to reconstruct different shapes within an object category (such as furniture) from unaligned 2D images.

By lifting 3D points into a higher-dimensional canonical space, the network can represent radical structural differences—such as the difference between a solid stool and an armchair with gaps and holes—as projections of a single, continuous function. The network does not need to learn how to mathematically "break" a surface to create a hole; it simply learns to steer the continuous surface through a higher dimension where the hole already exists as a natural geometric property.

Key Foundational Work

Dynamic Multi-View Scene Reconstruction Using Neural Implicit Surface (2023) — contributed 2 concept elements to this direction.
Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera (2022) — contributed 2 concept elements to this direction.
DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid (2023) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_topological_discontinuity_modeling_da74a0d6,
  title = {topological discontinuity modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/topological-discontinuity-modeling/rdr_da74a0d618ee46c3a5cdce176f74d4a8},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_topological_discontinuity_modeling_da74a0d6_v1,
  title = {topological discontinuity modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/topological-discontinuity-modeling/rdr_da74a0d618ee46c3a5cdce176f74d4a8/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake