When we look at a chair, we don't see a monolithic block of pixels; we see a backrest, a seat, and four legs. This "compositional" view of the world is fundamental to human intelligence, allowing us to recognize a chair even if it has a different style of leg or a missing slat.
In computer vision, researchers have long sought to replicate this by building part-based models. However, a persistent bottleneck has plagued these efforts: the "Fixed-K" problem. Most traditional architectures require a human to pre-define exactly how many parts an object has. If you tell a model a bird has five parts, but the input image is a close-up of a beak, the model struggles to allocate its representational "budget."
Recent research is shifting toward dynamic part decomposition—systems that can automatically discover semantic components and adapt their complexity to the specific object or scene at hand.
The Problem with Static Hierarchies
The mathematical appeal of compositional models is their efficiency. Alan Yuille and Roozbeh Mottaghi demonstrated in Complexity of Representation and Inference in Compositional Models with Part Sharing that by sharing parts (like using the same "leg" model for horses and cows) and using an "executive summary" principle to coarsen detail at higher levels, models can recognize an exponential number of objects in linear time.
However, as noted by Han Zhang in the development of SPDA-CNN, these models often break down when faced with variable structural complexity. A bird might have a visible tail, wings, and beak, while a piece of complex machinery might have dozens of identifiable components. A model with a fixed number of "slots" for parts will either over-segment simple objects (splitting a single wing into three redundant parts) or under-segment complex ones (lumping the engine and the chassis together).
Discovering Parts Without Labels
To move away from manual part definitions, researchers are turning to unsupervised discovery. The intuition is that if a model is forced to reconstruct an object from a set of "building blocks," it will naturally learn to identify semantic parts that are consistent across different instances.
In Discovering 3D Parts from Image Collections, Chun-Han Yao and colleagues introduced Latent Part Discovery (LPD). By training a Variational Auto-Encoder (VAE) on geometric primitives, the model learns a "part prior." It realizes that a chair is most efficiently reconstructed as an assembly of interchangeable components. Similarly, Shidi Li and Miaomiao Liu developed EditVAE, which disentangles "style" from "pose" for individual parts. This allows the model to understand that a "leg" is still a "leg" whether it is straight or tapered, or whether the chair is upright or overturned.
The breakthrough in these unsupervised methods, such as the Masked Part Autoencoder (MPAE), is treating part discovery as a "fill-in-the-blanks" task. By masking regions of an image and forcing the model to restore them using a library of learned descriptors, the model aligns high-level semantic concepts (like a ship's hull) with low-level visual details.
Adapting to Variable Complexity
The most recent frontier involves models that don't just discover parts, but decide how many parts are needed for a specific input.
Kehan Li and Runyi Yu addressed this in ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation. Instead of using a fixed cluster count, ACSeg uses an Adaptive Concept Generator. This allows the model to discover a variable number of "concepts" tailored to the complexity of each image—using more concepts for a cluttered street scene and fewer for a portrait.
This dynamic approach is also being applied to the very "bones" of objects. In Category-Agnostic Neural Object Rigging (CANOR), Chen Geng and Jiajun Wu represent objects as a sparse set of "blobs." This allows the system to discover the rigging structure of any deformable object—from a fish to a pair of eyeglasses—without needing a pre-defined skeleton.
Why It Works: The Consistency Constraint
The reason these models can find "parts" without being told what they are is the consistency constraint. As explored in Cerberus by Boyang Deng and Geoffrey E. Hinton, if you force a model to explain an object's appearance across different viewpoints and poses using a set of components, the most "stable" explanation is usually the one that aligns with physical parts. A head always moves with the neck; a wheel always stays attached to the axle.
By moving away from fixed hyperparameters and toward these adaptive, consistency-driven architectures, computer vision is moving closer to a human-like understanding of the physical world: not as a grid of pixels, but as a flexible assembly of meaningful pieces.