MCBoost: Multiple Classifier Boosting for Perceptual Co-clustering of Images and Visual Features

Tae-Kyun Kim, Roberto CipollaNeurIPS 2008 · OpenReview

The Challenge of Multi-Modality in Object Detection

In computer vision, real-world objects are rarely uniform in appearance. A human face looks radically different from the front than it does in profile, and a pedestrian's appearance changes drastically based on their clothing, posture, or the ambient lighting. This high variance in appearance is known as multi-modality.

Standard boosting algorithms, such as the famous Viola-Jones detector, typically attempt to find a single set of features to describe an entire object class. However, when the data is highly varied, a single "strong classifier" struggles to capture every variation effectively. The traditional workaround is to manually label sub-categories (e.g., "front-face," "side-face") and train separate detectors for each. Unfortunately, manual labeling is tedious, prone to human error, and often fails to reflect the most "perceptually" or mathematically distinct features of the dataset.

To address this, the authors introduce a novel co-clustering problem: how can an algorithm group similar images together while simultaneously finding the specific visual features that define those groups?

The MCBoost Framework

The core contribution of this paper is MCBoost (Multiple Classifier Boosting). Instead of training a single classifier to recognize an entire highly varied dataset, the system trains $K$ "strong classifiers" (experts) simultaneously.

MCBoost Pipeline: The framework takes a set of object images, visual features, and random background images, outputting distinct perceptual clusters alongside their most discriminative features.

The framework relies on several key mechanisms to achieve this co-clustering:

  • Noisy-OR Aggregation: The authors utilize a "Noisy-OR" logic to combine the experts. A test image is classified as an "object" if any of the $K$ experts identifies it as such. This structure allows each expert to specialize in a specific niche (e.g., one expert for profile views, another for frontal views) without being penalized for failing to recognize objects outside its expertise.
  • Competitive Learning: During training, the classifiers actively compete for samples. If a specific classifier is slightly better at identifying a certain subset of images, the algorithm updates the weights so that this classifier becomes the designated "expert" for those images. This naturally pushes classifiers to ignore samples they struggle with and double down on their strengths.
  • Concurrent Clustering and Feature Selection: Unlike standard k-means, which clusters images based on raw pixel similarity, MCBoost clusters images based on their discriminative features. The image clusters and their defining visual features are learned in the same iterative loop.
  • Risk-Map Pruning: To maintain computational efficiency, the authors employ a "risk map" to identify areas in the image where the current model is most likely to fail. They then restrict the search for new visual features to these high-risk areas, significantly accelerating the training process.

Experimental Validation and Key Insights

The authors evaluated MCBoost on two primary benchmarks: the INRIA Pedestrian Dataset (featuring people in various poses and backgrounds) and the CMU PIE Face Dataset (featuring faces across 9 poses and 5 lighting conditions). Thousands of random non-object images were used as negative background samples.

Superior Accuracy and Intuitive Grouping

On both datasets, MCBoost significantly outperformed standard AdaBoost, demonstrating much lower false-negative rates for the same number of false positives.

More importantly, the algorithm produced highly intuitive groupings. Standard unsupervised clustering often produces "random-looking" groups because it lacks context on which parts of an image are important. MCBoost, however, produces "perceptual" clusters by grouping images based on the features that actually help distinguish them from the background.

Intuitive Grouping: MCBoost automatically separates images into distinct clusters based on perceptual similarities like lighting and pose.

In the face experiments, MCBoost automatically grouped images by pose (e.g., left profile vs. frontal) and lighting (e.g., shadows on the left vs. right), even though it was never explicitly programmed to understand what a pose or a shadow was. Surprisingly, MCBoost even performed better than an AdaBoost model trained on manually labeled poses, suggesting that the algorithm finds more efficient, mathematically sound data partitions than human intuition.

Efficiency

Despite the complexity of training multiple classifiers simultaneously, the "risk map" approach allowed the system to remain computationally feasible. An unoptimized Matlab implementation was able to scan an image and classify over 172,000 patches in just 3.6 seconds.

Limitations and Future Directions

While powerful, MCBoost does have a few limitations. The user must manually define the number of classifiers ($K$) beforehand. Although the authors suggest starting with a large $K$ and pruning redundant clusters, an automated method for determining the optimal number of experts would improve the framework. Additionally, while the method is robust, it benefits from a good initialization (like a quick k-means pass), as purely random starting weights can lead to slower convergence.

Ultimately, MCBoost presents a compelling approach to handling multi-modal data. The authors suggest that this co-clustering logic could be applied to domains far beyond simple object detection, potentially serving as a valuable tool for broader data exploration and "perceptual" data mining.

Summary updated

Abstract

We present a new co-clustering problem of images and visual features. The problem involves a set of non-object images in addition to a set of object images and features to be co-clustered. Co-clustering is performed in a way of maximising discrimination of object images from non-object images, thus emphasizing discriminative features. This provides a way of obtaining perceptual joint-clusters of object images and features. We tackle the problem by simultaneously boosting multiple strong classifiers which compete for images by their expertise. Each boosting classifier is an aggregation of weak-learners, i.e. simple visual features. The obtained classifiers are useful for multi-category and multi-view object detection tasks. Experiments on a set of pedestrian images and a face data set demonstrate that the method yields intuitive image clusters with associated features and is much superior to conventional boosting classifiers in object detection tasks.
BibTeX
@inproceedings{kim2008mcboost,
  author = {Tae-Kyun Kim and Roberto Cipolla},
  title = {{MCBoost: Multiple Classifier Boosting for Perceptual Co-clustering of Images and Visual Features}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2008},
  url = {https://openreview.net/forum?id=rJb868Zd-B},
}
Figure