Multimodal Object Recognition Using Random Clustering Trees

Michael Villamizar, Anais Garrell, Alberto Sanfeliu, Francesc Moreno-NoguerIbPRIA 2015 · DOI

TL;DR

This paper introduces a novel approach for object recognition that can automatically discover and distinguish between different visual appearances (modalities) within a single object class. By combining Boosted Random Ferns (BRFs) for robust classification with Probabilistic Latent Semantic Analysis (PLSA) for clustering, the method efficiently learns to recognize objects with complex, multi-view appearances without requiring extensive manual annotation.

What Problem Does This Paper Address?

Recognizing objects in images and videos is a fundamental challenge in computer vision. While significant progress has been made, a key difficulty lies in handling intra-class variations, meaning objects of the same category can look very different due to factors like viewpoint, lighting, or pose. Traditional methods often tackle this by training separate classifiers for each distinct appearance, which leads to increased complexity and computational cost. Furthermore, manually annotating training data to identify these different appearance clusters is a tedious and time-consuming process. This paper aims to address the need for a more efficient and automated way to learn object detectors that can inherently distinguish between these multiple, visually distinct modes within a single class.

Methods

The proposed approach is a three-stage process:

  1. Object Tracking: An "online classifier" based on extremely randomized trees (specifically, random ferns) is used to detect and track the target object through an image sequence. This process is initiated with a single manual annotation (a bounding box in the first frame) and then self-learns by updating its hypotheses on subsequent frames. This stage automatically extracts a set of training samples of the object in various appearances.
  2. Object Classifier (BRFs): A more robust classifier, Boosted Random Ferns (BRFs), is trained using the collected training samples. BRFs combine multiple randomized trees using an AdaBoost algorithm to select the most discriminative trees, leading to an efficient and reliable object detector.
  3. Clustering (PLSA): To discover and distinguish between different intra-class modalities, the paper employs Probabilistic Latent Semantic Analysis (PLSA). PLSA, typically used for text analysis, is adapted here to cluster the training samples based on their visual features and the responses from the BRFs classifier. Essentially, it treats image features as "words" and object samples as "documents" to find latent "topics" (clusters) that represent distinct visual appearances of the object. This stage is performed with weak supervision, meaning it doesn't require explicit labeling of which cluster each training sample belongs to.

Datasets

The paper mentions the use of:

  • Synthetic datasets: Generated at random with varying numbers of clusters (K) and feature dimensions (D) to evaluate the method's performance in controlled environments.
  • Real datasets:
    • Two face sequences from a dataset proposed in [12] for face detection and pose estimation.
  • A toy car dataset for object recognition from multiple viewpoints.

Key Insights and Ideas

  • Weak Supervision for Clustering: The core innovation is using PLSA to automatically discover and group visually similar object appearances (clusters) without requiring manual annotation of these clusters during training. This significantly reduces the annotation burden.
  • BRFs for Robustness and Efficiency: Leveraging Boosted Random Ferns provides a powerful and efficient classifier that can handle complex visual data.
  • Unified Framework: The paper demonstrates that a single BRFs classifier, combined with PLSA-based clustering, can effectively handle multimodal object recognition, avoiding the need for multiple specialized classifiers.
  • Robotics Applications: The ability to distinguish object views is highlighted as particularly useful for robotics, enabling more context-aware actions and interactions.

Key Results

  • Synthetic Experiments: The proposed BRFs+PLSA approach achieved high classification rates (low Equal Error Rates - EER) and good separability between classes in synthetic 2D scenarios. It outperformed K-means in clustering accuracy, producing lower confusion values, especially with higher dimensional features.
  • Real Experiments:
    • Face Detection: The method successfully detected faces and estimated their pose, with different colors indicating different pose clusters.
  • Object Recognition (Toy Car): The approach was able to discretize the toy car's appearance into multiple modalities (K=5 clusters), with each cluster corresponding to a distinct viewpoint.

Limitations and Future Work

  • Limitations: The paper acknowledges that while the BRFs classifier is efficient, the PLSA clustering step can be computationally intensive. The method relies on an initial manual annotation for object tracking.
  • Future Work: The authors suggest that this approach is a step towards more sophisticated human-robot interaction, where understanding specific object views can lead to more intelligent actions. The paper implies that further exploration into integrating this into more complex robotic tasks is a natural next step.

Summary updated

BibTeX
@inproceedings{villamizar2015multimodal,
  author = {Michael Villamizar and Anais Garrell and Alberto Sanfeliu and Francesc Moreno-Noguer},
  title = {{Multimodal Object Recognition Using Random Clustering Trees}},
  booktitle = {IbPRIA},
  year = {2015},
  doi = {10.1007/978-3-319-19390-8\_56},
}
Figure