Abstract
Deep Implicit Functions (DIFs) have transformed 3D shape modeling by representing surfaces as neural network level sets. However, standard DIFs typically model object categories in isolation, leading to physical inconsistencies like interpenetration when applied to complex, multi-object environments such as the human abdomen. We present MODIF (Multi-Object Deformed Implicit Field), a novel framework designed to model multiple non-rigid objects simultaneously. By introducing cross-category signals and a specialized attraction-repulsion loss, MODIF ensures that reconstructed objects respect spatial boundaries and can even infer missing structures based on the surrounding anatomy.
Introduction: The "Lonely" DIF Problem
While DIFs excel at capturing the geometry of single objects, they struggle in dense environments where objects are packed tightly together. In anatomical modeling, the shape of an organ is not just a product of its own geometry but is also defined by the pressure and spatial constraints of its neighbors. Standard independent models often suffer from interpenetration—where two organs occupy the same physical space—or fail to maintain consistent spatial relationships. Furthermore, in clinical settings where scans may be partial or obscured, isolated models cannot leverage the context of surrounding structures to "fill in the blanks."
Methodology
MODIF moves away from modeling scenes as single global fields or independent collections. Instead, it employs a modular architecture that balances individual object fidelity with global scene consistency.
1. Category-Specific Sub-functions
The framework assigns a dedicated sub-network to each object category (e.g., liver, stomach, spleen). Each sub-network utilizes a shared template field, a hyper-network for instance-specific details, and a deformation field to manage non-rigid anatomical variations.
2. Cross-Category Refinement
The core innovation of MODIF is the refinement module ($U$). This module collects feature signals from all sub-functions simultaneously. By processing this global context, it generates a "correction field" that adjusts the boundaries of every object based on the presence and shape of its neighbors. This allows the model to treat the scene as a cohesive puzzle where the pieces dynamically adapt to one another.
3. Attraction-Repulsion Loss
To explicitly prevent physical overlaps, the authors introduced a specialized loss function. This function penalizes points where the implicit fields of two different objects both indicate an "inside" state. This forces the surfaces to meet at a clean interface rather than bleeding into one another, significantly reducing intersection volume.
Experimental Evaluation
The model was evaluated on three challenging medical benchmarks involving non-rigid anatomical structures:
- WORD: A large-scale dataset of abdominal organs (liver, spleen, pancreas, stomach, and kidneys).
- AbdomenCT: Used to test consistency across different scanning sources.
- MMWHS (Multi-Modality Whole-Heart Segmentation): Focused on heart structures with high contact and complex interfaces.
Key Results and Insights
- Superior Accuracy and Collision Avoidance: MODIF consistently outperformed state-of-the-art methods like DeepSDF and NDF. On the WORD dataset, MODIF reduced the Intersection Volume (IV)—a measure of overlap—by more than 60% compared to standard approaches.
- Missing-Object Recovery: One of the most significant findings was the model's ability to "hallucinate" missing organs. In experiments where a specific structure (like the left myocardium) was removed from the input, MODIF used the positions of surrounding organs to predict the missing shape with high accuracy, outperforming competitors by a factor of three in terms of Chamfer Distance.
- Dense Correspondence: By deforming a template for each organ, the model automatically establishes point-to-point correspondence across different patients, which is essential for medical atlas building and longitudinal tracking.
Conclusion
MODIF demonstrates that multi-object modeling benefits significantly from cross-category communication. By treating the anatomy as an interrelated system rather than a collection of parts, the framework achieves physically plausible, collision-free reconstructions. Future work may involve incorporating physics-based constraints, such as tissue elasticity, to further enhance the biological realism of the deformations.
Summary updated