Texture-based Tracking in mm-wave Images

Peter Salz, Gerd Reis, Didier StrickerVLUDS 2011 · OpenReview

TL;DR

This paper tackles the challenge of tracking objects in low-quality mm-wave images, where traditional color, intensity, and edge-based methods fail. They adapt and significantly enhance the Visual Tracking Decomposition (VTD) framework by introducing a novel texture-based region descriptor and improving motion models and sampling strategies, achieving superior performance on mm-wave data.

What Problem Does This Paper Address?

The paper addresses the critical problem of robust object tracking in challenging imaging conditions, specifically mm-wave imagery from full-body scanners used for airport security. These images are inherently low-quality due to low resolution, low contrast, and significant noise and artifacts. Traditional tracking algorithms, which heavily rely on color, intensity, and edge features, perform poorly in such environments. The goal is to accurately track potential threats on a person's body, which requires reliable tracking even when faced with severe appearance and illumination changes, shadowing, and bright metallic reflections.

Methods

The authors build upon the Visual Tracking Decomposition (VTD) framework, a particle-filter-based multi-template tracking method. Their key contributions include:

  • Novel Texture-based Region Descriptor: Instead of relying on standard features, they introduce a descriptor based on texture features, inspired by ultrasound image analysis. This descriptor captures information from spatial and frequency domains, including Kolmogorov-Smirnov distance of cumulative distributions, standard deviation, and various power spectrum characteristics.
  • Patch Decomposition: To compute texture features effectively, they divide the image region into five overlapping patches, with a dominant weight assigned to the central patch.
  • Enhanced Observation Model: They improve VTD's observation model by incorporating a sophisticated weighting scheme for observations and using a "representative sample" (the template closest to the median) instead of the model mean.
  • Improved Motion Model: The motion model is enhanced to better predict object movement by considering previous movements and adapting variance based on the difference between current and previous positions.
  • Sophisticated Sampling and Interaction: They employ a "multiple-try Metropolis-Hastings" algorithm for sampling, which increases confidence in proposed samples and reduces iterations. The tracker interaction scheme is also refined for more realistic state evaluation and contribution weighting.
  • Sparse Principal Component Analysis (SPCA): SPCA is used for feature reduction and to partition the observation model into sub-models for different basic trackers, aiming to capture severe appearance changes efficiently.

Datasets

  • Mm-wave data: Provided by Smith Heimann GmbH, these are sequences from full-body scanners showing a person rotating. Due to confidentiality, graphical examples are not shown, but the paper discusses their challenging characteristics (low resolution, low contrast, artifacts, severe appearance/illumination changes).
  • Normal-quality images: The authors also evaluated their method on standard image data, specifically mentioning sequences used by Kwon and Lee for the original VTD.

Key Insights and Ideas

  • Texture is Key for Low-Quality Images: The core insight is that texture-based features are far more robust than color, intensity, or edge features in challenging, low-quality imaging scenarios like mm-wave scans.
  • Beyond Standard Features: The success of their texture descriptor highlights the need to move beyond conventional feature sets when dealing with degraded image data.
  • Representative Sample Superiority: Replacing the model mean with a "representative sample" (closest to the median) in the observation model proves more effective for capturing the object's state in dynamic environments.
  • Adaptive Motion Prediction: A motion model that learns from past movements and adapts its variance based on observed displacement is crucial for accurate tracking in sequences with significant motion.
  • Controlled Sampling for Robustness: The multiple-try Metropolis-Hastings algorithm and refined tracker interaction lead to more reliable sample selection and a more stable tracking process, evidenced by more realistic acceptance rates.

Key Results

  • Superior Performance on Mm-wave Data: The proposed method significantly outperforms the original VTD on mm-wave sequences, which are described as extremely challenging.
  • Comparable Performance on Standard Data: On normal-quality images, their method achieves results comparable to, and sometimes even better than, the original VTD, though with a lower frame rate.
  • Realistic Acceptance Rates: The method achieves acceptance rates for state updates between 17% and 40%, which is theoretically more realistic than the very high rates observed in VTD.

Limitations and Future Work

  • Feature Computation Cost: The current feature computation is computationally expensive, requiring many Fourier transforms, which limits real-time performance.
  • Feature Set Optimization: A comparative study is needed to determine the minimal set of features required for optimal tracking quality.
  • Multi-Object Tracking: The current method is designed for single-object tracking. Extending it to track multiple objects simultaneously is a future direction.
  • Color Feature Integration: Incorporating color features is suggested to further improve tracking in colored image data.
  • Broader Application: The authors are confident that their texture-based descriptor can be extended to other challenging imaging modalities like infrared and ultrasound, and they plan to test and tune the method for these applications.

Summary updated

Abstract

Current tracking methods rely on color-, intensity-, and edge-based features to compute a description of an image region. These approaches are not well-suited for low-quality images such as mm-wave data from full-body scanners. In order to perform tracking in such challenging grayscale images, we propose several enhancements and extensions to the Visual Tracking Decomposition (VTD) by Kwon and Lee. A novel region descriptor, which uses texture-based features, is presented and integrated into VTD. We improve VTD by adding a sophisticated weighting scheme for observations, better motion models, and a more realistic way for sampling and interaction. Our method not only outperforms VTD on mm-wave data but also has comparable results on normal-quality images. We are confident that our region descriptor can easily be extended to other kinds of features and applications such that tracking can be performed in a large variety of image data, especially low-resolution, low-illumination and noisy images.
BibTeX
@inproceedings{salz2011texture,
  author = {Peter Salz and Gerd Reis and Didier Stricker},
  title = {{Texture-based Tracking in mm-wave Images}},
  booktitle = {VLUDS},
  year = {2011},
  url = {https://openreview.net/forum?id=4-MuePw7u0},
}
Figure