Abstract
Modeling multiple interacting non-rigid objects, such as internal organs, presents a significant challenge for traditional Deep Implicit Functions (DIFs), which typically represent objects in isolation. This paper introduces MODIF (Multi-Object Deformed Implicit Field), a novel framework designed to jointly model complex multi-object scenes. By incorporating cross-category refinement and a specialized attraction-repulsion loss, MODIF produces highly accurate, collision-free reconstructions and demonstrates the unique ability to "hallucinate" missing objects based on their anatomical context.
Introduction
While Deep Implicit Functions have advanced 3D shape representation, they often struggle with multi-object environments where physical interactions—such as pushing, squishing, and deforming—are prevalent. In medical imaging, modeling organs individually often leads to unrealistic "interpenetration" (overlapping surfaces), while modeling them as a single entity loses individual semantic identity. MODIF addresses this by treating the scene as a collection of interacting sub-functions that share information to maintain physical and anatomical consistency.
The MODIF Framework
MODIF decomposes a multi-object scene into $m$ category-specific sub-functions. Each sub-function learns a template shape and a deformation field to map that template to a specific instance. The architecture is built upon two primary innovations:
1. Cross-Category Refinement
Unlike independent networks, MODIF employs a global refinement module ($U$). This module aggregates feature signals from all objects in the scene to predict a joint correction field. This ensures that the deformation of one object (e.g., an expanding liver) informs the boundary adjustments of its neighbors (e.g., the stomach), maintaining realistic spatial relationships.
2. Attraction-Repulsion Loss
To explicitly manage contact between surfaces, the authors introduced a novel loss function ($\mathcal{L}^C$). This function acts as a spring system: it "attracts" surfaces to meet at contact points while "repelling" them to prevent interpenetration. This mechanism is critical for generating crisp, realistic boundaries between adjacent organs.
Experimental Evaluation
MODIF was evaluated on three major medical benchmarks: the Whole-body Organ Reconstruction Dataset (WORD), AbdomenCT-1K, and the Multi-Modality Whole-Heart Segmentation (MMWHS) dataset. It was also tested on a custom "Procedural Chairs" dataset to demonstrate generalizability.
Key Results
- Superior Accuracy: MODIF consistently achieved the lowest Chamfer Distance (CD) and Earth Mover’s Distance (EMD) across all medical datasets, outperforming state-of-the-art methods like DeepSDF and DIF-Net.
- Collision Mitigation: On the WORD dataset, MODIF reduced the Intersection Volume (IV)—a metric for surface overlap—by over 60% compared to the best-performing baselines.
- Robust Recovery: A standout feature of MODIF is its ability to reconstruct missing organs. By leveraging population-wide knowledge and the positions of surrounding organs, it can predict the shape and location of a missing heart structure with significantly higher accuracy than previous methods.
Conclusion
MODIF represents a significant step forward in the implicit modeling of complex scenes. By treating multi-object interaction as a primary signal rather than a secondary constraint, the framework achieves unprecedented levels of anatomical realism. Future work will explore the integration of biomechanical priors, such as tissue elasticity, to further enhance the physical fidelity of these digital twins.
Summary updated