Feature Voting for Object Localization via Density Ratio Estimation

Lian-Tao Wang, Dong Deng, Chunlei ChenKSII 2019 · DOI

Abstract

Support vector machine (SVM) classifiers have been widely used for object detection. These methods usually locate the object by finding the region with maximal score in an image. With bag-of-features representation, the SVM score of an image region can be written as the sum of its inside feature-weights. As a result, the searching process can be executed efficiently by using strategies such as branch-and-bound. However, the feature-weight derived by optimizing region classification cannot really reveal the category knowledge of a feature-point, which could cause bad localization. In this paper, we represent a region in an image by a collection of local feature-points and determine the object by the region with the maximum posterior probability of belonging to the object class. Based on the Bayes' theorem and Naive-Bayes assumptions, the posterior probability is reformulated as the sum of feature-scores. The feature-score is manifested in the form of the logarithm of a probability ratio. Instead of estimating the numerator and denominator probabilities separately, we readily employ the density ratio estimation techniques directly, and overcome the above limitation. Experiments on a car dataset and PASCAL VOC 2007 dataset validated the effectiveness of our method compared to the baselines. In addition, the performance can be further improved by taking advantage of the recently developed deep convolutional neural network features.

Introduction

In classical object detection, the "bag-of-features" approach combined with a Support Vector Machine (SVM) is a standard paradigm. While effective for image-level classification, it often falters in precise object localization. The core issue lies in the optimization objective: SVM weights are designed to maximize the margin between classes, not to accurately delineate object boundaries. This often results in "biased" weights where a positive bias leads to over-inclusion of background, and a negative bias causes the detected region to collapse into a small fragment of the object.

Methodology

The proposed framework reformulates localization as a Maximum Posterior probability problem. By applying Bayes’ theorem and Naive-Bayes assumptions, the task of finding the optimal region is transformed into maximizing the sum of individual feature scores.

Direct Density Ratio Estimation (DRE)

A critical innovation is the use of DRE to calculate feature scores. Instead of estimating the object-class likelihood and non-object-class likelihood separately—a task prone to error in high-dimensional spaces—the authors model the ratio directly. This ratio is represented as a linear combination of Gaussian kernels, with parameters solved by minimizing the Kullback–Leibler (KL) divergence.

Visualization of feature scores across an image, where individual points contribute to the overall localization score.

Search and Integration

The additive nature of the scoring function allows for efficient search strategies like Branch-and-Bound for rectangular boxes. Furthermore, the framework integrates modern deep features, such as "hyper-columns" from VGG-Net, to enhance semantic representation.

Experimental Results

The method was evaluated on the Campus Car and PASCAL VOC 2007 datasets.

Localization Performance

On the Car dataset, the DRE-based method achieved an Average Precision (AP) of 0.86, a significant improvement over the SVM-ESS baseline (0.78). When combined with deep CNN features, the performance reached a perfect AP of 1.0.

Localization results showing various candidate bounding boxes and their associated scores.

Efficiency and Irregular Shapes

The search process was notably faster, averaging 0.3s per image compared to 4.8s for SVM-based methods. The framework also proved capable of localizing irregular shapes via Branch-and-Cut, as the negative scores for background points naturally penalize expansion beyond object boundaries.

Conclusion

This research demonstrates that direct density ratio estimation provides a more robust foundation for object localization than traditional margin-based classification. While modern end-to-end DCNNs currently lead in raw speed, the statistical rigor and interpretability of the feature-voting framework offer valuable insights for future hybrid architectures.

Summary updated

Abstract

Support vector machine (SVM) classifiers have been widely used for object detection. These methods usually locate the object by finding the region with maximal score in an image. With bag-of-features representation, the SVM score of an image region can be written as the sum of its inside feature-weights. As a result, the searching process can be executed efficiently by using strategies such as branch-and-bound. However, the feature-weight derived by optimizing region classification cannot really reveal the category knowledge of a feature-point, which could cause bad localization. In this paper, we represent a region in an image by a collection of local feature-points and determine the object by the region with the maximum posterior probability of belonging to the object class. Based on the Bayes' theorem and Naive-Bayes assumptions, the posterior probability is reformulated as the sum of feature-scores. The feature-score is manifested in the form of the logarithm of a probability ratio. Instead of estimating the numerator and denominator probabilities separately, we readily employ the density ratio estimation techniques directly, and overcome the above limitation. Experiments on a car dataset and PASCAL VOC 2007 dataset validated the effectiveness of our method compared to the baselines. In addition, the performance can be further improved by taking advantage of the recently developed deep convolutional neural network features.
BibTeX
@inproceedings{wang2019feature,
  author = {Lian-Tao Wang and Dong Deng and Chunlei Chen},
  title = {{Feature Voting for Object Localization via Density Ratio Estimation}},
  booktitle = {KSII},
  year = {2019},
  doi = {10.3837/tiis.2019.12.012},
}
Figure