Real-time object retrieval with dominant orientation template matching improved by pyramid scoring

Chaoqun Hong, Jianke Zhu, Na Li, Jiajun Bu, Chun ChenJournal of Image and Graphics 2012 · DOI

Abstract

传统的基于内容的目标检索方法一般通过提取和比较局部特征,往往计算复杂度较高而且需要进行离线训练。因此,以最新的主方向模板(DOT)特征为基础提出新的实时目标检索方法。同时,利用搜索窗口中网格得分的空间分布情况构造相似度映射图,并采用多层次金字塔评分去除网格得分较为分散的错误窗口。利用以上改进实现的目标检索方法,真正能够无须离线训练即可从输入视频直接得到检索结果。实验结果表明,本文方法在提升检索性能的同时仍然保持主方向模板匹配的实时处理能力。

Introduction

Content-Based Object Retrieval (CBOR) is the task of identifying a specific object within a large image database or a video stream based on a user-provided query. Traditionally, this process involves extracting complex local features (such as SIFT or SURF) and comparing them against a massive index. This approach faces two primary bottlenecks: high computational complexity due to matching high-dimensional descriptors and the requirement for an offline training phase to build visual dictionaries or classifiers.

Researchers have developed Pyd-DOT, an improved real-time object retrieval method that eliminates the need for time-consuming offline training. By combining Dominant Orientation Templates (DOT) with a novel "pyramid scoring" strategy, the system can accurately find objects in video streams even in the presence of noise and partial occlusions, all while maintaining high processing speeds.

Methodology

The Pyd-DOT framework is built upon three key technical pillars designed to balance speed and robustness.

Dominant Orientation Templates (DOT)

Instead of complex descriptors, DOT divides an image into a grid and records only the strongest gradient direction in each cell. This representation is incredibly fast to compute and match using bitwise operations (SSE2 instructions), providing the foundation for real-time performance.

Likelihood Mapping and Filtering

To handle background noise, the authors create a "similarity map" for every search window. They apply a median filter to this map to suppress isolated noise—essentially "cleaning up" the signal before scoring it. This step helps prevent the system from being fooled by messy patches of background.

Pyramid Scoring Strategy

The core innovation of this research is the pyramid scoring strategy. Rather than just summing up all the scores in a window, the algorithm checks for spatial consistency across multiple levels of a pyramid. If high-scoring cells are scattered randomly (like confetti), the window is rejected as noise. If the high scores are clustered in a way that matches the object's structure, the window is kept. This ensures that the "good parts" of the match are actually where they are supposed to be relative to each other, providing significant robustness to partial occlusion.

Experimental Evaluation

The study evaluated the algorithm using the PROST dataset, a well-known benchmark for object tracking and retrieval consisting of four video sequences (Box, Liquor, Lemming, and Board) at 640x480 resolution. These videos include challenges like rapid movement, lighting changes, and partial occlusions.

Accuracy and Speed

The authors compared Pyd-DOT against the original DOT and the Efficient Subwindow Retrieval (ESR) method. Pyd-DOT consistently achieved higher PASCAL scores (a measure of bounding box overlap) compared to both baselines. In sequences like "Liquor" and "Lemming," where the original DOT often failed due to background similarity, Pyd-DOT successfully locked onto the target.

Crucially, the system remains firmly in the real-time category. While the original DOT processes at ~17.9 ms per frame, Pyd-DOT achieves approximately 25.8 ms per frame (roughly 38-40 FPS).

Conclusion and Future Work

Pyd-DOT represents a significant step forward for real-time applications by enabling "one-shot" retrieval without learned statistical models. By refining the matching process with spatial verification, the system achieves high accuracy without sacrificing the speed of template matching.

Future work will address limitations regarding sensitivity to large scale and rotation changes. Integrating more sophisticated color information or exploring adaptive templates that update as the object moves could further improve robustness in extremely dynamic environments.

Summary updated

BibTeX
@article{hong2012real,
  author = {Chaoqun Hong and Jianke Zhu and Na Li and Jiajun Bu and Chun Chen},
  title = {{Real-time object retrieval with dominant orientation template matching improved by pyramid scoring}},
  journal = {Journal of Image and Graphics},
  year = {2012},
  doi = {10.11834/jig.20120515},
}
Figure