Latent Constrained Correlation Filters for Object Localization

Ling Shao, Fellow of Ieee/iaprarXiv 2016 · arXiv

TL;DR

This paper introduces Latent Constrained Correlation Filters (LCCF), a novel approach to object localization that leverages "solution sampling" to improve robustness against noise and occlusion. By mapping correlation filters to a learned latent subspace and employing a new Subspace Alternating Direction Method of Multipliers (SADMM) algorithm, LCCF achieves state-of-the-art performance on eye localization and car detection tasks.

What Problem Does This Paper Address?

Traditional correlation filter methods for object localization, while efficient, often struggle with real-world challenges like occlusion, noise, and illumination variations. These issues degrade performance because standard training methods can lead to "solution sampling," where the learned parameters are overly sensitive to the specific training data. The authors observe that if we can make the "solution sampling" more predictable and controllable, we can improve the robustness of the learned models. This paper aims to address this by introducing a method that explicitly constrains the learning process within a learned subspace, making it more resilient to distortions.

Methods

The core innovation is the Latent Constrained Correlation Filters (LCCF) method. It builds upon existing correlation filter frameworks but introduces two key advancements:

  • Latent Subspace Constraint: Instead of directly learning filters, LCCF maps them to a learned latent subspace. This subspace is designed to capture variations present in the data, allowing the filters to be more robust. The learning framework embeds distribution-related constraints within this subspace.
  • Subspace Alternating Direction Method of Multipliers (SADMM): To efficiently solve the optimization problem introduced by the subspace constraint, the authors propose SADMM. This is a novel variant of the classical ADMM algorithm, specifically designed to handle the subspace constraint. SADMM is shown to converge at a saddle point, offering improved efficiency and theoretical convergence guarantees.

The training process involves initializing filters with a subset of data, then iteratively refining them by projecting onto the learned subspace and updating the subspace based on new samples.

Datasets

The paper evaluates LCCF on the following datasets:

  • CMU Multi-PIE: Used for eye localization experiments. It contains frontal faces with neutral expressions and normal illumination.
  • Labeled Faces in the Wild (LFW): Also used for eye localization experiments, offering greater diversity in ages, sexes, and races, as well as variations in lighting and pose.
  • MIT Streetscape: Used for car detection experiments, providing images of street scenes.

Key Insights and Ideas

  • Solution Sampling as a Feature: The paper reframes the issue of "solution sampling" not just as a problem, but as an opportunity. By understanding how data sampling influences solution parameters, they can design methods to control and leverage this effect for robustness.
  • Subspace Learning for Robustness: Projecting correlation filters onto a learned latent subspace is shown to be an effective strategy for improving robustness against noise and occlusion. This subspace implicitly captures variations, making the learned filters generalize better.
  • SADMM's Efficiency: The proposed SADMM algorithm is crucial for making the subspace constraint practical. It offers a significant speedup in convergence compared to traditional ADMM for this specific problem.

Key Results

  • Eye Localization: LCCF demonstrates superior performance compared to state-of-the-art methods like MCCF, CFwLB, ASEF, and MOSSE, particularly when images are affected by noise and occlusion.
  • Car Detection: LCCF also outperforms other correlation filters in car detection, showing significant robustness against noise and occlusion.
  • Speed: The paper claims SADMM converges twice as fast as the original ADMM. In testing, LCCF is noted to be very fast due to element-wise operations in the FFT domain. The training time complexity is O(NDlogD) per image, and memory cost is O(maxiterKD), which is considered efficient.

Limitations and Future Work

  • Limitations: The paper acknowledges that the subspace constraint, while beneficial, might introduce extra storage costs due to variable replication.
  • Future Work: The authors suggest applying their LCCF algorithm to other applications such as tracking, face recognition, and action recognition.

Summary updated

Abstract

There is a neglected fact in the traditional machine learning methods that the data sampling can actually lead to the solution sampling. We consider this observation to be important because having the solution sampling available makes the variable distribution estimation, which is a problem in many learning-related applications, more tractable. In this paper, we implement this idea on correlation filter, which has attracted much attention in the past few years due to its high performance with a low computational cost. More specifically, we propose a new method, named latent constrained correlation filters (LCCF) by mapping the correlation filters to a given latent subspace, in which we establish a new learning framework that embeds distribution-related constraints into the original problem. We further introduce a subspace based alternating direction method of multipliers (SADMM) to efficiently solve the optimization problem, which is proved to converge at the saddle point. Our approach is successfully applied to two different tasks inclduing eye localization and car detection. Extensive experiments demonstrate that LCCF outperforms the state-of-the-art methods when samples are suffered from noise and occlusion.
BibTeX
@misc{ieeeiapr2016latent,
  author = {Ling Shao, Fellow of IEEE/IAPR},
  title = {{Latent Constrained Correlation Filters for Object Localization}},
  year = {2016},
  eprint = {1606.02170},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1606.02170},
}
Figure