Implicit Modeling of Non-rigid Objects with Cross-Category Signals

Yuchun Liu, Benjamin Planche, Meng Zheng, Zhongpai Gao, Pierre Sibut-Bourde, Fan Yang, Terrence Chen, Ziyan Wu

Abstract

Deep Implicit Functions (DIFs) have transformed 3D shape modeling by representing surfaces as neural network level sets. However, standard DIFs typically model object categories in isolation, leading to physical inconsistencies like interpenetration when applied to complex, multi-object environments such as the human abdomen. We present MODIF (Multi-Object Deformed Implicit Field), a novel framework designed to model multiple non-rigid objects simultaneously. By introducing cross-category signals and a specialized attraction-repulsion loss, MODIF ensures that reconstructed objects respect spatial boundaries and can even infer missing structures based on the surrounding anatomy.

Introduction: The "Lonely" DIF Problem

While DIFs excel at capturing the geometry of single objects, they struggle in dense environments where objects are packed tightly together. In anatomical modeling, the shape of an organ is not just a product of its own geometry but is also defined by the pressure and spatial constraints of its neighbors. Standard independent models often suffer from interpenetration—where two organs occupy the same physical space—or fail to maintain consistent spatial relationships. Furthermore, in clinical settings where scans may be partial or obscured, isolated models cannot leverage the context of surrounding structures to "fill in the blanks."

The MODIF pipeline demonstrates the progression from a generic template surface to a refined, collision-aware reconstruction through deformation and cross-category correction.

Methodology

MODIF moves away from modeling scenes as single global fields or independent collections. Instead, it employs a modular architecture that balances individual object fidelity with global scene consistency.

1. Category-Specific Sub-functions

The framework assigns a dedicated sub-network to each object category (e.g., liver, stomach, spleen). Each sub-network utilizes a shared template field, a hyper-network for instance-specific details, and a deformation field to manage non-rigid anatomical variations.

2. Cross-Category Refinement

The core innovation of MODIF is the refinement module ($U$). This module collects feature signals from all sub-functions simultaneously. By processing this global context, it generates a "correction field" that adjusts the boundaries of every object based on the presence and shape of its neighbors. This allows the model to treat the scene as a cohesive puzzle where the pieces dynamically adapt to one another.

The MODIF architecture: Category-specific sub-networks generate initial fields, which are then refined by a cross-category module (U) that integrates signals from all objects to produce the final, consistent SDF fields.

3. Attraction-Repulsion Loss

To explicitly prevent physical overlaps, the authors introduced a specialized loss function. This function penalizes points where the implicit fields of two different objects both indicate an "inside" state. This forces the surfaces to meet at a clean interface rather than bleeding into one another, significantly reducing intersection volume.

Experimental Evaluation

The model was evaluated on three challenging medical benchmarks involving non-rigid anatomical structures:

  • WORD: A large-scale dataset of abdominal organs (liver, spleen, pancreas, stomach, and kidneys).
  • AbdomenCT: Used to test consistency across different scanning sources.
  • MMWHS (Multi-Modality Whole-Heart Segmentation): Focused on heart structures with high contact and complex interfaces.

Key Results and Insights

  • Superior Accuracy and Collision Avoidance: MODIF consistently outperformed state-of-the-art methods like DeepSDF and NDF. On the WORD dataset, MODIF reduced the Intersection Volume (IV)—a measure of overlap—by more than 60% compared to standard approaches.
  • Missing-Object Recovery: One of the most significant findings was the model's ability to "hallucinate" missing organs. In experiments where a specific structure (like the left myocardium) was removed from the input, MODIF used the positions of surrounding organs to predict the missing shape with high accuracy, outperforming competitors by a factor of three in terms of Chamfer Distance.
  • Dense Correspondence: By deforming a template for each organ, the model automatically establishes point-to-point correspondence across different patients, which is essential for medical atlas building and longitudinal tracking.

Conclusion

MODIF demonstrates that multi-object modeling benefits significantly from cross-category communication. By treating the anatomy as an interrelated system rather than a collection of parts, the framework achieves physically plausible, collision-free reconstructions. Future work may involve incorporating physics-based constraints, such as tissue elasticity, to further enhance the biological realism of the deformations.

Summary updated

Abstract

Deep implicit functions (DIFs) have emerged as a potent and articulate means of representing 3D shapes. However, methods modeling object categories or non-rigid entities have mainly focused on single-object scenarios. In this work, we propose MODIF, a multi-object deep implicit function that jointly learns the deformation fields and instance-specific latent codes for multiple objects at once. Our emphasis is on non-rigid, non-interpenetrating entities such as organs. To effectively capture the interrelation between these entities and ensure precise, collision-free representations, our approach facilitates signaling between category-specific fields to adequately rectify shapes. We also introduce novel inter-object supervision: an attraction-repulsion loss is formulated to refine contact regions between objects. Our approach is demonstrated on various medical benchmarks, involving modeling different groups of intricate anatomical entities. Experimental results illustrate that our model can proficiently learn the shape representation of each organ and their relations to others, to the point that shapes missing from unseen instances can be consistently recovered by our method. Finally, MODIF can also propagate semantic information throughout the population via accurate point correspondences.
BibTeX
@inproceedings{liu2024implicit,
  author = {Yuchun Liu and Benjamin Planche and Meng Zheng and Zhongpai Gao and Pierre Sibut-Bourde and Fan Yang and Terrence Chen and Ziyan Wu},
  title = {{Implicit Modeling of Non-rigid Objects with Cross-Category Signals}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i4.28171},
}
Figure