Introduction
Democratizing 3D design through natural language is a major goal in computer vision. However, current models often struggle with "entanglement." For instance, an instruction to make a chair's legs thinner might inadvertently alter the backrest or add unwanted structural elements. Previous state-of-the-art methods, such as ChangeIt3D, often became overly conservative to avoid these artifacts, resulting in edits so subtle they were barely perceptible.
LADIS (Language Disentanglement for 3D Shape Editing) addresses this by grounding language in 3D geometry such that independent semantic concepts are represented independently in the model's latent space. This allows for localized, obvious, and semantically accurate edits without requiring explicit part-based supervision.
Methodology
The LADIS framework employs a three-pronged approach to ensure precise shape manipulation:
- LADIS Loss (Language Disentanglement): This regularization loss forces the embeddings of independent edit instructions to be orthogonal. By ensuring that descriptions of different parts or attributes do not overlap in the latent space, the model naturally learns to produce geometrically local edits.
- Multi-Expert Shape-Difference Encoder: The architecture utilizes multiple "expert" modules specializing in different geometric features (e.g., dimensions, texture, structure). A voting network dynamically selects the most relevant experts for a given text prompt.
- Iterative Editing Procedure:
- Neighborhood Simplex Editing (NSE): To maintain object realism, the search space is restricted to a manifold defined by the nearest neighbors of the original shape.
- Output-Driven Edit Step-Size Adjustment (ODESSA): This mechanism automatically rescales optimization steps to ensure every edit results in a consistent, noticeable visual change.
Evaluation and Results
LADIS was evaluated using the ShapeTalk and ShapeGlot datasets. The model demonstrated significant improvements across several metrics:
- Edit Locality: LADIS outperformed previous state-of-the-art methods by 20% in Part-wise Edit Precision (PEP).
- Accuracy: It achieved a 6.6% improvement in language reference resolution.
- Human Perception: In a study involving 500 annotations, humans rated LADIS edits as 72% more semantically accurate and 61% more localized than previous leading methods.
Qualitative results confirm that LADIS successfully produces "obvious" changes—such as clearly thinner legs—while leaving irrelevant parts of the object completely untouched.
Conclusion
By focusing on language disentanglement, LADIS provides a robust solution for localized 3D shape editing. The model moves beyond the trade-off between accuracy and visibility, enabling precise, text-driven design. Future work will look toward expanding the linguistic scope beyond English and applying these techniques to higher-fidelity 3D meshes and complex modalities.
Summary updated