A Novel 3D Gradient LBP Descriptor for Action Recognition

Zhaoyang Guo, Xin'an Wang, Bo Wang, Zheng XieIEICE Trans. Information and Systems 2017 · DOI

Introduction

Action recognition—the ability for a computer to identify human activities in video—is a cornerstone of modern computer vision. Most traditional methods rely on Spatio-Temporal Interest Points (STIPs), which are patches of motion within a video. However, existing descriptors for these patches often face a trade-off between information loss and computational cost. Many 2D-based descriptors ignore the rich 3D relationships across space and time, while high-dimensional 3D descriptors like HOG-HOF are often too slow for real-time deployment.

Researchers have addressed these challenges by developing a novel 3D Gradient Local Binary Pattern (LBP) descriptor. This method extends traditional image textures into the 3D spatio-temporal volume of a video, achieving high accuracy while remaining computationally efficient and compact.

Methodology: The 3D Gradient LBP

The process follows a standard pipeline of detection, description, and classification, but introduces a sophisticated new way to describe video patches. After detecting STIPs and extracting 3D cuboids, the algorithm applies the 3D Gradient LBP:

Overview of the 3D Gradient LBP descriptor construction, illustrating the six-plane approach and binary decomposition.
  1. The Six-Plane Approach: Instead of looking at just three orthogonal planes, the algorithm considers six planes surrounding a central feature: front, rear, left, right, above, and below. This captures the "grey-level space" distribution much more effectively than previous iterations.
  2. Thresholding and Gradient: It compares the average value of each plane to the central feature using a threshold ($T$), capturing the intensity gradient in all directions. Using averages makes the descriptor naturally resistant to video noise.
  3. Compact Encoding: To prevent exponential growth in dimensionality, the authors split the result into two binary parts. This reduces the descriptor to a compact 128 dimensions, making it easier to store and search.

Experimental Results

The method was evaluated on three classic action recognition benchmarks: the KTH Dataset (basic actions), the Weizmann Dataset (varied movement classes), and the UT Interaction Dataset (human-to-human interactions).

Performance Comparison

The 3D Gradient LBP proved to be a "best of both worlds" solution, offering high speed and compactness without sacrificing accuracy.

Comparison of the proposed method against state-of-the-art descriptors in terms of feature length, processing time, and accuracy across datasets.

As shown in the table, the proposed method is significantly faster than heavyweights like 3D SIFT, processing in just 0.0078 seconds. Despite its small 128-dimension size, it achieved competitive accuracy across all datasets, including 92.25% on KTH and 91.42% on the complex UT Interaction dataset.

Conclusion and Future Work

By using six planes instead of three, the 3D Gradient LBP captures spatio-temporal context more effectively than previous LBP methods. Its efficiency through decomposition allows it to remain discriminative without the high dimensionality that plagues other 3D features.

While the descriptor performs excellently on standard benchmarks, future research may focus on combining these low-level features with "mid-level" semantic features or integrating these 3D gradient insights into deep learning architectures (such as 3D CNNs) to better handle highly cluttered or occluded scenes.

Summary updated

BibTeX
@article{guo2017novel,
  author = {Zhaoyang Guo and Xin'an Wang and Bo Wang and Zheng Xie},
  title = {{A Novel 3D Gradient LBP Descriptor for Action Recognition}},
  journal = {IEICE Trans. Information and Systems},
  year = {2017},
  doi = {10.1587/transinf.2017edl8006},
}
Figure