Name That Part: 3D Part Segmentation and Naming

Soumava Paul, Prakhar Kaushik, Ankit Vaidya, Anand Bhattad, Alan L. Yuille

Introduction

In the field of 3D computer vision, identifying that a group of points belongs together—known as class-agnostic segmentation—has seen significant progress. However, the subsequent step of assigning meaningful semantic labels to these parts (e.g., distinguishing an "airplane wing" from an "engine cowling") remains a major hurdle. Existing approaches typically fall into two categories: they either produce unlabeled geometric clusters or rely on query-based systems that cannot decompose an entire object simultaneously. Furthermore, inconsistent labeling across datasets—where a "leg" on a chair differs fundamentally from a "leg" on a tripod—complicates the learning process.

ALIGN-Parts aims to segment and name parts of 3D objects, such as the casing and dial of a telephone or the fuselage and thrusters of a spaceship.

The ALIGN-Parts framework addresses this "naming gap" by treating part segmentation as a direct set-alignment problem. It introduces a feed-forward architecture that simultaneously segments and names parts in a single pass, offering a significant leap in both accuracy and speed.

The ALIGN-Parts Framework

The core innovation of ALIGN-Parts is the Partlet. Instead of assigning a label to every individual point in a 3D cloud, the model decomposes the shape into a set of implicit representations called partlets. Each partlet captures both a spatial segmentation mask and a semantic embedding.

The pipeline consists of three critical stages:

  1. BiCo Fusion (Feature Fusion): To accurately identify parts, the model must understand both geometry and appearance. It combines 3D geometric features with high-level appearance features from DINOv2 using a localized bi-directional cross-attention mechanism. This ensures the model recognizes that a part's "texture" is as informative as its "shape."
  2. Partlet Generation: A decoder learns a fixed number ($K$) of adaptive part-level representations. These partlets aggregate the fused point features into cohesive, functional units.
  3. Semantic Grounding via Bipartite Matching: To assign names, the authors utilize Large Language Models (LLMs) to generate affordance-aware descriptions. Rather than using simple labels like "handle," the system uses descriptions such as "the part of a door grasped to open it." These descriptions are embedded using MPNet and matched to the predicted partlets using the Sinkhorn algorithm for optimal transport. This set-to-set matching allows the model to handle a variable number of parts per object dynamically.

Dataset Harmonization and TexParts

A significant contribution of this work is the cleanup and unification of existing 3D data. The authors harmonized taxonomies across major datasets including PartNet, 3DCoMPaT++, and Find3D, resulting in a unified ontology of 1,794 unique 3D parts.

Leveraging ALIGN-Parts as a scalable annotation engine, they created TexParts, a new dataset derived from TexVerse. TexParts contains 8,450 objects spanning 14,000 part categories, providing a rich resource for future research in fine-grained 3D understanding.

Key Results and Efficiency

ALIGN-Parts demonstrates state-of-the-art performance, outperforming baselines like PartField by 15.8% in class-agnostic mIoU. Beyond accuracy, the model's feed-forward nature provides a massive efficiency boost. By avoiding the iterative clustering required by previous methods, ALIGN-Parts is approximately 100x faster at inference time (~0.05s vs. ~4s).

Comparison of ALIGN-Parts against the PartField baseline and its application in new dataset annotation. ALIGN-Parts provides coherent semantic labels where previous methods struggle.

This speed translates directly to practical utility. In real-world annotation tests, the model enabled humans to annotate 3D objects 5–8x faster than manual methods, reducing the time per object from over 15 minutes to under 5 minutes while maintaining high quality.

Conclusion and Future Work

By shifting the focus from simple labels to affordance-based descriptions and employing a set-alignment architecture, ALIGN-Parts bridges the gap between 3D geometry and semantic meaning. While the model currently faces challenges with out-of-distribution objects and relies on frozen backbones, it sets a new standard for 3D part segmentation. Future research will explore end-to-end training and extending these capabilities to articulated objects, which is crucial for applications in robotics and autonomous manipulation.

Summary updated

BibTeX
@inproceedings{paul2026name,
  author = {Soumava Paul and Prakhar Kaushik and Ankit Vaidya and Anand Bhattad and Alan L. Yuille},
  title = {{Name That Part: 3D Part Segmentation and Naming}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2026},
  url = {https://openreview.net/forum?id=tBNSkuTQUZ},
}
Figure