TL;DR
S2MDF introduces a lightweight, plug-and-play layer that enforces a hard non-interpenetration constraint on multi-object implicit 3D representations. By projecting vector-valued Signed Distance Fields (SDFs) onto a valid geometric manifold, it eliminates physically impossible object overlaps down to numerical precision without requiring network architecture changes or sacrificing reconstruction quality.
What Problem Does This Paper Address?
Implicit Neural Representations (INRs), particularly Signed Distance Fields (SDFs), have become the gold standard for modeling continuous 3D geometry. However, when moving from single objects to multi-object scenes or multi-part composite shapes, scaling up becomes problematic. Typically, each object or part is assigned its own independent SDF. Because these fields do not communicate with one another, they frequently produce physically impossible configurations where multiple objects interpenetrate—occupying the exact same space.
Existing mitigation strategies rely on "soft" penalty terms added to the loss function during training. While these penalties reduce the severity of overlaps, they fail to eliminate them entirely, require tedious hyperparameter tuning for loss weights, and offer absolutely no guarantees at inference time.
Methods
The authors formalize the problem by defining a Multi-object Distance Field (MDF), which introduces a hard mathematical constraint on the vector of SDF values $\mathbf{u}$ at any given point in space:
$$\min^{(1)}(\mathbf{u}) + \min^{(2)}(\mathbf{u}) \ge 0$$
This constraint dictates that the sum of the two smallest SDF values must be non-negative. This mathematically ensures that at most one object can have a negative distance value (i.e., be "inside") at any single spatial location.
To enforce this, the authors introduce S2MDF, a lightweight layer appended to the end of an existing network. They propose two distinct projection mechanisms to map unconstrained SDF outputs onto the valid MDF manifold:
- Quadratic Programming (QP): Formulates the projection as a standard differentiable QP problem. While mathematically optimal in an $L_2$ sense, its computational cost scales quadratically with the number of objects ($K$).
- Shift-All (Analytical Heuristic): A fast, alternative mapping that simply subtracts half of the constraint violation from all fields when an intersection occurs. This approach preserves the relative ordering and pairwise differences of the original fields.
Crucially, the authors prove that this sum-of-mins formulation is closed under linear interpolation. If the constraint is satisfied at discrete grid vertices, standard meshing algorithms like Marching Cubes are theoretically guaranteed to extract completely intersection-free meshes.
Datasets
The method was validated across several diverse datasets representing different domains:
- TS-Lung: A component of the TotalSegmentator dataset containing 318 lung CT scans split into 5 distinct lobes, which frequently exhibit multi-way intersections.
- MM-WHS: A whole-heart segmentation dataset consisting of 20 CT samples with 7 highly complex, tightly packed cardiac components.
- Replica: An indoor scene reconstruction dataset featuring 8 distinct real-world room layouts containing dozens of individual objects.
- PartNet / ShapeNet: Used for part-based object reconstruction, specifically focusing on subsets of 100 chairs (8 parts each) and 100 mixers (4 parts each).
Key Insights and Ideas
- The Interpolation Trap: A naive constraint like $\min^{(2)}(\mathbf{u}) \ge 0$ works perfectly at discrete sample points but fails to prevent intersections between points once the field is linearly interpolated during meshing (see
figure:art_30d7fd765ba84b3f8f7cdce7b92a9d80). The core theoretical insight of this paper is that the sum-of-mins constraint is concave, meaning linear interpolation inherently preserves the non-intersection property everywhere. - Optimization vs. Heuristics: Surprisingly, the fast "Shift-All" heuristic performs just as well as, and sometimes better than, the mathematically optimal QP projection. This occurs because the $L_2$ metric optimized by QP is arbitrary, whereas Shift-All preserves the structural gradients (pairwise differences) of the underlying fields.
- Zero-Shot Post-Processing: Because S2MDF is a pure geometric projection layer, it can be applied as a post-processing step at inference time on already trained models, instantly wiping out intersections without requiring any retraining.
Key Results
- Zero Intersections: S2MDF successfully reduces intersection volumes by several orders of magnitude—dropping from typical baseline levels of $10^{-3}$ down to numerical precision ($10^{-7}$ to $10^{-12}$), effectively eliminating overlaps entirely.
- Preserved Reconstruction Quality: Standard geometric metrics like Chamfer Distance (MCD) and F1 scores remain stable or show slight improvements, proving that enforcing physical plausibility does not degrade surface accuracy.
- Zero Computational Overhead: While the QP approach doubled training times for complex objects, the Shift-All variant introduced no measurable computational overhead during training or inference, making it highly scalable.
Limitations and Future Work
- Numerical Precision Limits: At strict mathematical boundaries ($\epsilon = 0$), floating-point errors can still cause microscopic intersections during meshing. While adding a tiny margin ($\epsilon = 10^{-4}$) completely resolves this issue, it introduces a minor hyperparameter.
- Independent Meshing: Currently, standard meshing algorithms are still run on each object's projected SDF independently. While the resulting meshes do not intersect, they do not explicitly share topology or vertices at their contact boundaries. The authors note that designing multi-object-aware meshing algorithms that natively handle shared boundaries is an important avenue for future research.
Summary updated