Curvature: A signature for Action Recognition in Video Sequences

He Chen, Gregory S. ChirikjianWorkshops @ CVPR 2020 · DOI

Introduction

Modern action recognition typically relies on deep learning models that require vast amounts of annotated data. These models often process videos as a series of discrete frames, sometimes missing the continuous temporal flow of an action. This research proposes a geometric alternative: treating an entire video sequence as a single curve in a high-dimensional pixel space. By analyzing the intrinsic geometric properties of this path—specifically its curvature and torsion—researchers can create a compact, data-efficient signature for human actions.

Methodology: Video as a Geometric Path

The fundamental premise is that a video sequence of $N$ frames, each with $d \times d$ pixels, can be viewed as a trajectory in a $d^2$-dimensional Euclidean space.

Conceptual overview of action recognition as a trajectory analysis in a high-dimensional space.

Signal Alignment and Normalization

To account for varying speeds of the same action, the authors employ the Globally Optimal Reparameterization Algorithm (GORA). This step aligns sequences by reparameterizing them based on their arclength in pixel space, effectively normalizing the "tempo" of the motion.

Curvature and Torsion Extraction

Once aligned, the geometric properties of the curve are calculated using the Frenet frame. The method focuses on:

  1. First Curvature ($\kappa$): Measures how sharply the curve bends.
  2. Second Curvature ($\tau$ or Torsion): Measures how much the curve twists out of a local plane.
The first curvature distributions provide a distinct signature for different action classes.

Statistical Indexing

Rather than using the raw curvature values, the authors extract statistical descriptors (mean, variance, skewness, and kurtosis). They also introduce a Beta index to better characterize multimodal distributions where standard central moments might overlap. These compact signatures are then classified using a Random Forest.

Experimental Results

The approach was validated on two benchmarks: the Weizmann Dataset (controlled environment) and the UCF Sports Dataset (complex backgrounds).

  • Data Efficiency: The geometric signature allows for high accuracy even with small training sets, addressing the "data thirst" of traditional deep learning.
  • Accuracy: On the Weizmann dataset, the algorithm achieved 95.0% accuracy. On the more challenging UCF Sports dataset, incorporating Mask R-CNN for subject segmentation pushed accuracy to 87.0%.
  • The Role of Torsion: Ablation studies confirmed that including the second curvature (torsion) is vital, as it captures 3D-like evolution of the motion, significantly boosting recognition rates over using first curvature alone.

Discussion and Future Work

The curvature-based approach offers inherent invariance to mirror reflections and rigid-body displacements. However, it faces challenges with "subtle motions" where sensor noise can interfere with the signal. Future work aims to fuse these geometric signatures with other modalities, such as audio, to enhance scene understanding in robotics.

Summary updated

BibTeX
@inproceedings{chen2020curvature,
  author = {He Chen and Gregory S. Chirikjian},
  title = {{Curvature: A signature for Action Recognition in Video Sequences}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00437},
}
Figure