Deep Boosting: Layered Feature Mining for General Image Classification

Zhanglin Peng, Liang Lin, Ruimao Zhang, Jing Xu

Abstract

Constructing effective representations is a critical but challenging problem in multimedia understanding. The traditional handcraft features often rely on domain knowledge, limiting the performances of exiting methods. This paper discusses a novel computational architecture for general image feature mining, which assembles the primitive filters (i.e. Gabor wavelets) into compositional features in a layer-wise manner. In each layer, we produce a number of base classifiers (i.e. regression stumps) associated with the generated features, and discover informative compositions by using the boosting algorithm. The output compositional features of each layer are treated as the base components to build up the next layer. Our framework is able to generate expressive image representations while inducing very discriminate functions for image classification. The experiments are conducted on several public datasets, and we demonstrate superior performances over state-of-the-art approaches.

Introduction

In the mid-2010s, computer vision was transitioning from hand-crafted features (like SIFT or HoG) to learned representations. While deep learning (CNNs) showed immense promise, these models were often seen as "black boxes" with massive parameter counts that were difficult to interpret or control. Traditional Boosting algorithms (like Adaboost) were excellent at selecting the most relevant features from a pool but were limited by the simplicity of their "weak classifiers." A standard regression stump can only do so much when faced with complex image data. This research addresses this by asking: Can we use the iterative selection power of boosting to build a deep, hierarchical feature representation from scratch?

Methods

The proposed "Deep Boosting" framework operates in a layer-wise manner to bridge the gap between traditional boosting and hierarchical deep learning.

The Deep Boosting architecture showing layer-wise feature composition and regression stumps.

Layer-wise Stacking and Feature Construction

  1. Preprocessing: Images are decomposed using Gabor wavelets (8 orientations, 1 scale) to serve as "visual primitives."
  2. Discriminative Feature Selection: In each layer, a modified Gentle Adaboost algorithm is used. Instead of standard binary thresholds, a sigmoid function is introduced into the regression stumps to create smoother, more optimizable weak classifiers.
  3. Composite Feature Construction: After selecting the most discriminative features in one layer, the model combines them in pairs based on spatial proximity to form "composite features." These composites then serve as the input candidates for the next layer.
  4. Multi-class Decision: The system uses a "one-against-all" strategy, training a strong binary classifier for every category in the dataset.

Key Insights

Instead of just using Boosting to pick features, it is used to grow an architecture. Each layer's output is a more sophisticated version of the previous layer's input. Spatial constraints are crucial; the model ensures that the hierarchy captures local structures that make sense in a 2D image.

Experimental Results

The researchers evaluated their approach on standard benchmarks: Caltech 256 (Easy10 and Var10 subsets) and the 15 Scenes Dataset.

Performance Comparison

The "Deep Boosting" method consistently outperformed then-state-of-the-art methods like Sparse Coding (ScSPM) and Locality-constrained Linear Coding (LLC):

  • Caltech 256 (Easy10): Achieved 94.99% accuracy (vs. LLC's 93.61%).
  • Caltech 256 (Var10): Achieved 85.45% accuracy, a significant jump over ScSPM (70.11%).
  • 15 Scenes: Reached 81.76% average precision, outperforming GIST+SVM (75.12%).

The Impact of Depth

A key finding of the study is that classification accuracy improved consistently as the model moved from Layer 1 to Layer 3 across all tested categories, validating the hierarchical approach.

Classification accuracy improvement across layers for various datasets.

Conclusion and Future Work

The Deep Boosting framework demonstrates that expressive image representations can be generated by inducing discriminative functions in a layer-wise manner. However, limitations remain. Training is relatively slow, taking 5–9 hours per category on contemporary hardware. Because it uses a one-against-all strategy, training time scales linearly with the number of classes. Future work could explore learning the initial primitives themselves rather than using fixed Gabor filters and optimizing inference speed, which currently ranges from 25–40 seconds per image.

Summary updated

Abstract

Constructing effective representations is a critical but challenging problem in multimedia understanding. The traditional handcraft features often rely on domain knowledge, limiting the performances of exiting methods. This paper discusses a novel computational architecture for general image feature mining, which assembles the primitive filters (i.e. Gabor wavelets) into compositional features in a layer-wise manner. In each layer, we produce a number of base classifiers (i.e. regression stumps) associated with the generated features, and discover informative compositions by using the boosting algorithm. The output compositional features of each layer are treated as the base components to build up the next layer. Our framework is able to generate expressive image representations while inducing very discriminate functions for image classification. The experiments are conducted on several public datasets, and we demonstrate superior performances over state-of-the-art approaches.
BibTeX
@inproceedings{peng2014deep,
  author = {Zhanglin Peng and Liang Lin and Ruimao Zhang and Jing Xu},
  title = {{Deep Boosting: Layered Feature Mining for General Image Classification}},
  booktitle = {ICME},
  year = {2014},
  url = {https://openreview.net/forum?id=Kqm8WF10fjR},
}
Figure