Actionness-Assisted Recognition of Actions

Ye Luo, Loong-Fah Cheong, An TranICCV 2015 · DOI

Introduction

Traditional action recognition frameworks often struggle with background clutter and complex multi-agent interactions. By treating video as independent frames or relying on rigid spatial grids, these methods frequently fail to distinguish between intentional biological motion and irrelevant environmental movement. This paper proposes the concept of "actionness"—a generic, precategorical measure of intentional movement that identifies where an action is occurring before attempting to classify what it is.

Methodology: Building the Actionness Map

The proposed framework represents video through temporal superpixels—long-term trajectories of image patches. From these trajectories, four distinct "actionness" attributes are extracted to capture the essence of biological agency:

  1. Sudden Change (SC): Captures ballistic movements (e.g., kicking) by measuring abrupt shifts in a trajectory’s speed, direction, or size.
  2. Repetitive Motion (RM): Identifies rhythmic patterns (e.g., walking) using Mutual Information (MI) to find periodicities within a single trajectory.
  3. Temporal Synchrony (TS): Groups coordinated movements between different trajectories, even if they are spatially distant. This is vital for recognizing interactions like fencing or hugging where agents move in concert.
  4. Image Saliency (Sa): Incorporates standard visual saliency to highlight prominent static features that stand out visually.
Overview of the Actionness framework, illustrating the fusion of various motion cues into a unified actionness map.

These cues are fused into a single Actionness Map, which guides Actionness-driven Pooling. Instead of uniform spatial pyramids, the system pools features specifically from regions with high actionness, focusing the model's attention on intentional movement rather than background noise.

Experimental Evaluation

The method was evaluated across several benchmarks for both localization (detection) and classification (recognition):

  • Action Detection: On the UCF-Sports dataset, the framework achieved a Mean Average Precision (mAP) of 66.81, significantly outperforming previous state-of-the-art methods like L-CORF (60.8) and DPM (54.9).
  • Action Recognition: The actionness-driven pooling yielded substantial gains. On the SSBD dataset (focused on rhythmic self-stimulatory behaviors), accuracy reached 76.0%, a massive jump over the 44.0% baseline.
  • Large-Scale Benchmarks: On the HMDB51 dataset, the model achieved 60.41% accuracy, showing strong performance in categories involving complex human-to-human interaction.

Discussion and Conclusion

Actionness provides an agent-agnostic approach to motion analysis, mimicking human perception by grouping objects based on coordination and "common fate" rather than just physical proximity. While highly effective for large-scale biological motion, the model currently faces challenges with subtle facial actions and high computational costs associated with Mutual Information calculations. Future work aims to improve efficiency through hashing schemes to enable real-time trajectory indexing and localization.

Summary updated

BibTeX
@inproceedings{luo2015actionness,
  author = {Ye Luo and Loong-Fah Cheong and An Tran},
  title = {{Actionness-Assisted Recognition of Actions}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2015},
  doi = {10.1109/iccv.2015.371},
}
Figure