Optimization Framework for Learning a Hierarchical Shape Vocabulary for Object Class Detection

Sanja Fidler, Marko Boben, Ales LeonardisBMVC 2009 · DOI

TL;DR

This paper introduces a novel optimization framework for learning hierarchical shape vocabularies for object detection. By iteratively refining layers from both bottom-up and top-down perspectives, their method achieves more compact vocabularies, faster inference, and improved detection accuracy compared to previous approaches.

What Problem Does This Paper Address?

Object class detection, a fundamental task in computer vision, often relies on visual vocabularies to represent objects. While flat vocabularies are common, hierarchical approaches offer advantages in computational efficiency and qualitative performance by capturing structural dependencies between object parts. However, existing methods for learning these hierarchical vocabularies often suffer from limitations: they are typically learned in a strictly bottom-up manner, leading to potential loss of discriminative shape information or overfitting to generic parts. Furthermore, the selection of vocabulary shapes is often based solely on frequency, which can be suboptimal. This paper addresses the challenge of learning a compact and discriminative hierarchical shape vocabulary that effectively captures object structure without explicit supervision of part locations.

Methods

The core contribution is a stochastic optimization framework for learning hierarchical shape vocabularies. Unlike previous methods that learn layers sequentially and in a fixed bottom-up fashion, this approach iteratively refines the hierarchy. The learning process is two-fold:

  • Bottom-up phase: This phase learns and selects vocabulary shapes for a given layer, aiming to maximize its expressiveness while keeping complexity low. This is achieved through a greedy selection followed by a stochastic MCMC (Metropolis-Hastings) optimization.
  • Top-down phase: This phase improves the vocabulary of a lower layer by incorporating feedback from the layer above. It identifies "critical points" in the data that are poorly explained by the current hierarchy and uses these points to re-learn shapes in the lower layer, aiming to enhance the performance of the higher layer.

The algorithm iterates between these two phases for consecutive layers, allowing for a more globally optimal vocabulary. The overall objective is to optimize a function that balances the log-likelihood of the data with a penalty for model complexity.

Datasets

The paper evaluates its approach on 11 diverse object classes from standard recognition datasets:

  • ETH shape data set (Apple logo, bottle, giraffe, mug, swan)
  • GRAZ dataset (bicycle_side, car_front, cow_side)
  • INRIA horses
  • TUD motorbike
  • UIUC multi-scale car_side

Key Insights and Ideas

  • Iterative Bottom-Up and Top-Down Refinement: The most significant insight is the introduction of a top-down feedback mechanism to refine bottom-up learned vocabularies. This addresses the limitation of strictly bottom-up methods that can get stuck in local optima.
  • "Critical Points" for Targeted Learning: Identifying data points that are poorly explained by the current hierarchy is a clever way to focus learning on the most informative aspects of the data, leading to more discriminative shape models.
  • Balancing Expressiveness and Complexity: The optimization objective that penalizes model complexity is crucial for learning compact and efficient vocabularies, which directly translates to faster inference.
  • Compositional Nature of Objects: The work leverages the inherent compositional structure of objects, where complex shapes are built from simpler parts, to build a hierarchical representation.

Key Results

The proposed approach demonstrates significant improvements over the previous method [6] and achieves competitive results with state-of-the-art methods:

  • Vocabulary Size: Approximately 3 times more compact.
  • Inference Speed: 2.5 times faster.
  • Detection Performance: 10% higher detection performance.
  • Training Time: While the full optimization approach takes 5 times longer than the baseline, it is still competitive with other state-of-the-art methods. For example, training on the cow, mug, and giraffe classes took 23-31 minutes.

Limitations and Future Work

  • Training Time: The paper acknowledges that the iterative optimization process leads to longer training times compared to a single bottom-up pass.
  • Optimization Scope: While the framework is general, the authors found that optimizing two consecutive layers at a time yielded satisfactory results, suggesting that optimizing more layers simultaneously might be computationally prohibitive or offer diminishing returns.
  • Future Work: The paper implies that further exploration of optimizing more layers simultaneously or applying the framework to different types of visual features could be fruitful directions. The authors also mention that the learned object models are cross-validated to discard those yielding too many false positives, hinting at the importance of validation in refining the final vocabulary.

Summary updated

BibTeX
@inproceedings{fidler2009optimization,
  author = {Sanja Fidler and Marko Boben and Ales Leonardis},
  title = {{Optimization Framework for Learning a Hierarchical Shape Vocabulary for Object Class Detection}},
  booktitle = {BMVC},
  year = {2009},
  doi = {10.5244/c.23.93},
}

Loading related content...

Figure