In computer vision, we often treat objects as monolithic entities—a "chair," a "plane," or a "bird." However, the physical world is modular. A chair is a composition of a seat, a backrest, and legs. Traditionally, teaching a machine to understand this decomposition required "segmentation masks": painstaking, human-labeled maps where every pixel or 3D point is assigned a part label.
A growing body of research suggests that this manual labor might be unnecessary. Instead of being told where parts are, models can "discover" them by solving other tasks, such as reconstructing a shape from a limited set of blocks or learning to describe an object in a language game. This phenomenon, known as unsupervised semantic part discovery, reveals that parts are not just human conventions—they are the most efficient way for a neural network to represent the structure of the world.
The Primitive Bottleneck
The most intuitive way to force a model to discover parts is to give it a "box of blocks." If you ask a neural network to reconstruct a complex 3D mesh but only allow it to use a few simple geometric primitives—like cuboids or spheres—it must decide which parts of the object are important enough to warrant a block.
Shubham Tulsiani and Hao Su demonstrated in Learning Shape Abstractions by Assembling Volumetric Primitives that when a model is trained to minimize the difference between a target shape and an assembly of boxes, it naturally learns a consistent "alphabet" of parts. For example, the model might always use "Primitive #3" to represent the seat of a chair, regardless of the chair's specific style. This consistency emerges because using the same primitive for the same functional component across a dataset is the most efficient way to minimize reconstruction error.
Researchers have since expanded this "alphabet" to be more expressive. Despoina Paschalidou and Andreas Geiger moved beyond simple boxes to "superquadrics"—mathematical shapes that can morph from rounded pebbles to sharp-edged blocks. In Superquadrics Revisited: Learning 3D Shape Parsing Beyond Cuboids, they showed that these flexible primitives allow models to capture more organic parts, like the wings of an airplane or the limbs of an animal, without any human guidance.
Emergence from Self-Supervision
Parts also emerge when models are trained on massive amounts of data without any specific geometric constraints. In these cases, the "discovery" happens within the internal layers of the network.
When Maxime Oquab and colleagues trained DINOv2: Learning Robust Visual Features without Supervision, they found that the model's attention maps naturally clustered around semantic parts. Even though the model was only trained to match images to other similar images, it learned that the "wing" of a bird and the "wing" of an airplane share a similar structural role.
This happens because parts are "invariant" features. While the overall shape of a chair changes, the relationship between a flat surface (seat) and vertical supports (legs) is a constant pattern. Qing Su and Shihao Ji observed in FLSL: Feature-level Self-supervised Learning that as information flows deeper into a network, these features naturally refine into distinct semantic partitions. Even generative models like 3D-GAN, developed by Jiajun Wu and Bill Freeman, spontaneously develop "semantic neurons" that fire only when the model is generating a specific component, like a chair leg.
Grounding Parts in Language and Logic
A third path to discovery is through "grounding"—linking 3D geometry to other modalities like language or engineering logic.
In PartGlot: Learning Shape Part Segmentation from Language Reference Games, Juil Koo and Minhyuk Sung set up a "reference game." One agent describes an object (e.g., "the chair with the tall backrest") and another agent must pick it out. To succeed, the model must learn which geometric regions correspond to the word "backrest." The semantic boundaries of the parts emerge as a byproduct of the need to communicate.
Similarly, models can discover parts by trying to "reverse-engineer" how an object would be built in CAD software. Daxuan Ren and Jianfei Cai developed ExtrudeNet and CSG-Stump to decompose point clouds into "sketches" and "extrusions." By forcing the model to explain a shape through engineering operations (like adding or subtracting volumes), the model discovers parts that are not just visually distinct, but also functionally and mechanically logical.
Why Discovery Works
The success of these diverse methods—from 3D Point Capsule Networks by Tolga Birdal to PARTICLE by Oindrila Saha—points to a fundamental truth in computer vision: semantic parts are the "low-rank" basis of the visual world.
Whether the objective is to compress a shape into boxes, distinguish it from other shapes in a dataset, or describe it to a human, the most efficient solution is to identify the underlying parts. By removing the need for manual labels, these unsupervised methods allow us to scale 3D understanding to the vast, unlabeled "wild" of the internet, creating models that see the world not as a soup of points, but as a structured assembly of meaningful components.