Shape Recognition from Fragmented and Occluded Contours

33 concepts31 papers1 proposal

Imagine trying to recognize a coffee mug on a cluttered desk. If a book covers the handle, a computer vision system relying on a global template or a complete silhouette will likely fail. Because a global descriptor aggregates information across the entire object, changing even a small fraction of the pixels—or introducing background clutter—completely alters the resulting mathematical representation.

To recognize objects in the real world, computer vision systems must be robust to occlusion, shadows, and sensor noise. A powerful research direction addresses this by shifting from global shape descriptions to semi-local and primitive-based feature matching. By breaking a shape down into independent, localized geometric fragments, these techniques ensure that when an object is partially hidden, the visible pieces remain perfectly recognizable.

The Principle of Local Support

The core insight behind this research direction is the concept of local support. If a feature descriptor's mathematical definition depends only on a small, isolated neighborhood of a shape, it becomes invariant to changes occurring outside that neighborhood.

David H. Marimont laid the groundwork for this perspective in A Representation for Image Curves, demonstrating that representing shapes through a discrete set of critical points provides local support. Because the representation of a visible segment is decoupled from the rest of the curve, an occlusion elsewhere on the object leaves the remaining fragments entirely unchanged.

This strategy applies equally to 2D contours and 3D surfaces. For instance, when matching 3D facial scans that may be partially obscured or incomplete, Yueming Wang et al. introduced the Sphere-Spin-Image in Sphere-Spin-Image: A Viewpoint-Invariant Surface Representation for 3D Face Recognition. By restricting the surface descriptor to a local spherical neighborhood, the authors made the representation independent of global geometry, allowing successful partial-to-whole surface matching.

Global Descriptor (Brittle)      Semi-Local Primitives (Robust)
     [   Cat Silhouette   ]           [Ear]  [Paw]  [Tail]
               │                               │      │      │
       (Tail is occluded)                      (Tail is occluded)
               ▼                               ▼      ▼      ▼
     [ Completely Changed ]           [Ear]  [Paw]  [XXXX]
     (Matching Fails)                 (Ear & Paw still match!)

Mariano Tepper et al. formalized this distinction in Morphological Shape Context: Semi-locality and Robust Matching in Shape Recognition by treating individual boundaries as independent entities. In traditional shape contexts, the descriptor encodes the global distribution of all points in a scene, meaning that adding background clutter or a second object distorts the descriptor of the target object. By focusing on independent, "meaningful boundaries," semi-local approaches prevent peripheral scene changes from corrupting the target object's representation.

Decomposing Shapes into Geometric Primitives

To operationalize local support, a continuous shape must be broken down into discrete, manageable components. Early model-based vision systems, such as the one proposed by W. A. Perkins in Model-Based Vision System for Scenes Containing Multiple Parts, segmented object boundaries into geometric primitives like straight lines and circular arcs (termed "concurves"). When an object is partially obscured, the system maintains high localization accuracy simply by matching the remaining visible primitives to the structural model.

This strategy of primitive decomposition has proven highly effective across diverse applications:

  • Texture-less Object Detection: Smooth, untextured objects lack distinct point features like SIFT corners. To overcome this, Federico Tombari et al. designed the Bunch Of Lines Descriptor (BOLD) in BOLD Features to Detect Texture-less Objects, which aggregates local geometric relationships between neighboring line segments. Similarly, Stevica Cvetković et al. demonstrated in DETECTION OF TEXTURE-LESS OBJECTS BY LINE-BASED APPROACH that a system can recover an object's full 3D pose from just a single correctly matched line primitive, allowing for successful identification even when 90% of the surface is obscured.
  • Boundary Reconstruction: In urban remote sensing, Yun Shi et al. utilized partial geometric arcs in MANHOLE COVER DETECTION USING VEHICLE-BASED MULTI-SENSOR DATA to reconstruct full circular object boundaries from fragmented visual data caused by heavy shadows and vehicle occlusions.
  • Deformable Object Tracking: For non-rigid objects, Xiaolin Zhao et al. showed in Non-rigid Object Tracking as Salient Region Segmentation and Association that decomposing an object's contour into short boundary "edgelets" provides sufficient structural evidence to maintain a precise tracking mask even when portions of the object temporarily disappear.

Overcoming Fragmented and Noisy Edges

In real-world imagery, edge detectors rarely produce clean, continuous lines. Shadows, highlights, and low contrast cause edges to appear broken, noisy, or truncated. If a system requires a perfectly sequenced boundary to calculate a shape signature, it will fail.

To solve this, researchers developed descriptors that are invariant to the length and starting point of a contour fragment. Hayko Riemenschneider et al. addressed this in Using Partial Edge Contour Matches for Efficient Object Category Localization by introducing a "self-containing" angular descriptor. This mechanism allows arbitrary, noisy sub-sections of an edge to be matched directly to a global reference model without requiring a complete or perfect edge extraction.

From a mathematical perspective, Shuo Feng et al. tackled the sequencing problem in Classification of curves in 2D and 3D via affine integral signatures. By partitioning a curve into local segments based on its lowest-order affine invariant and applying integral signatures, the authors ensured that shape matching succeeds even when the starting point of the curve is completely unknown or parts of the boundary are missing.

Resolving Ambiguity through Structural Inference

While local primitives provide robustness to occlusion, they introduce a new challenge: ambiguity. A single straight line or a simple arc could belong to almost any object. How does a system know that a handful of isolated fragments actually constitute the target object?

The solution lies in verifying the global geometric and structural relationships between those fragments. Michael O. Shneier pioneered this in Recognition Using Semantic Constraints, using the geometric context between small surface fragments to iteratively filter out impossible interpretations. If the spatial relationships between the observed fragments match the expected structure of the model, a confident detection can be triggered.

This balance between local evidence and global consistency is achieved through several distinct mechanisms:

  1. Partial Triggering Thresholds: In Block-Based Search Space Reduction Technique for Face Detection Using Shoulder and Head Curves, Thambipillai Srikanthan et al. implemented a multi-part structural matching strategy where only a subset of expected geometric segments is required to trigger a detection window, maintaining high recall in cluttered scenes. Similarly, Dong Li et al. demonstrated in Object Detection in Image with Complex Background that matching specific hull boundary fragments rather than entire templates successfully localizes vessels docked against complex, cluttered shorelines.
  2. Internal Contour Modeling: To handle self-occlusion (such as limbs crossing in front of a torso), Greg Mori et al. incorporated internal edges and contours into shape descriptors in Estimating Human Body Configurations Using Shape Context Matching. This part-based composition strategy allows the system to "mix and match" individual limb fragments to reconstruct complex 3D poses.
  3. Message-Passing Networks: Bypassing explicit edge extraction entirely, Edward Hsiao et al. introduced Gradient Networks: Explicit Shape Matching Without Extracting Edges. By treating shape matching as a path-finding problem and passing messages across low-level image gradients, their framework integrates fragmented local evidence into a globally consistent path, maintaining high accuracy even when objects are up to 50% occluded.
  4. Global Optimization and Verification: Early neural approaches, such as the recurrent network proposed by Volker Tresp in A Neural Network Approach for Three-Dimensional Object Recognition, resolve local ambiguity by finding the global match that minimizes structural constraints between 2D features and a 3D model. In dynamic tracking scenarios, Swagat Kumar et al. showed in An occlusion reasoning scheme for monocular pedestrian tracking in dynamic scenes that when global optimization fails during total occlusions, local interest point matching (such as SURF) serves as a vital verification step to re-identify targets the moment they emerge.

By shifting the representational burden from the whole to the parts, semi-local feature matching transforms shape recognition from a fragile template-matching exercise into a robust process of geometric puzzle-solving.

Key Foundational Work

Model-Based Vision System for Scenes Containing Multiple Parts (1977) — contributed 2 concept elements to this direction.
Model-Based Vision System for Scenes Containing Multiple Parts (1977) — contributed 1 concept elements to this direction.
A Representation for Image Curves (1984) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_occlusion_robust_feature_matching_4a64b972,
  title = {occlusion-robust feature matching},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/occlusion-robust-feature-matching/rdr_4a64b97241164e9f86a18e5819eeaea7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_occlusion_robust_feature_matching_4a64b972_v1,
  title = {occlusion-robust feature matching},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/occlusion-robust-feature-matching/rdr_4a64b97241164e9f86a18e5819eeaea7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake