Abstract
Recently spatial pyramid matching (SPM) with scale invariant feature transform (SIFT) descriptor has been successfully used in image classification. Unfortunately, the codebook generation and feature quantization procedures using SIFT feature have the high complexity both in time and space. To address this problem, in this paper, we propose an approach which combines local binary patterns (LBP) and three-patch local binary patterns (TPLBP) in spatial pyramid domain. The proposed method does not need to learn the codebook and feature quantization processing, hence it becomes very efficient. Experiments on two popular benchmark datasets demonstrate that the proposed method always significantly outperforms the very popular SPM based SIFT descriptor method both in time and classification accuracy.
Introduction
In the early 2010s, the gold standard for image classification was the "Bag of Visual Words" (BoW) model using SIFT descriptors. While powerful, this approach has two major drawbacks. First, standard BoW treats an image like an unordered collection of features, ignoring spatial relationships. Second, the computational bottleneck is severe; turning SIFT features into "visual words" requires codebook generation (clustering millions of features) and quantization (mapping features to the nearest word), both of which are demanding on CPU and memory. While Spatial Pyramid Matching (SPM) addressed the spatial issue by partitioning images into grids, it still relied on the slow SIFT-based pipeline. The authors sought a method that retains spatial information but avoids the expensive codebook requirement.
Methods: The ComSPM Framework
The proposed framework, ComSPM (Combined Spatial Pyramid Matching), replaces SIFT with training-free descriptors that produce discrete codes immediately.
LBP and TPLBP Descriptors
Instead of SIFT, the method utilizes Local Binary Patterns (LBP) and Three-Patch Local Binary Patterns (TPLBP). LBP compares a pixel to its neighbors to capture micro-textures, while TPLBP compares small patches of pixels to capture more complex local structures.
Spatial Pyramid Representation and Fusion
The image is divided into increasingly fine grids (e.g., $1\times1$, $2\times2$, and $4\times4$). Histograms of the LBP and TPLBP codes are calculated for every cell. The core novelty is the weighted combination of these two descriptors:
$$ComSPM = \lambda \cdot LBPSPM + (1 - \lambda) \cdot TPLBPSPM$$
By setting $\lambda = 0.3$, the method balances fine-grained texture detail with structural information. The final concatenated histograms are fed into a Support Vector Machine (SVM) using a histogram intersection kernel.
Experimental Results
The researchers evaluated ComSPM on two benchmarks: the 15-Class Scene Category dataset and Caltech 101. The proposed method outperformed the SIFT-SPM baseline in every metric:
- Accuracy: On the 15-class scene dataset, ComSPM reached 82.68% accuracy (vs. 81.38% for SIFT). On Caltech 101, it reached 65.50% (vs. 64.06% for SIFT).
- Speed: The average processing time per image was slashed from 1.28 seconds to just 0.56 seconds—a speedup of roughly 2.3x.
- Simplicity: The LBP component alone is incredibly fast, taking only 0.11 seconds per image, making it a viable ultra-fast alternative.
Conclusion
ComSPM demonstrates that a complex visual vocabulary is not strictly necessary for high-performance image classification. By preserving spatial layout with simple, mathematically defined descriptors like LBP and TPLBP, the method achieves state-of-the-art results with significantly lower computational costs. Future work may involve adaptive weighting of descriptors or incorporating sparse coding to further boost accuracy.
Summary updated