Action and Interaction Recognition in First-Person Videos

Sanath Narayan, Mohan S. Kankanhalli, Kalpathi Ramakrishnan

Abstract

In this work, we evaluate the performance of the popular dense trajectories approach on first-person action recognition datasets. A person moving around with a wearable camera will actively interact with humans and objects and also passively observe others interacting. Hence, in order to represent real-world scenarios, the dataset must contain actions from first-person perspective as well as third-person perspective. For this purpose, we introduce a new dataset which contains actions from both the perspectives captured using a head-mounted camera. We employ a motion pyramidal structure for grouping the dense trajectory features. The relative strengths of motion along the trajectories are used to compute different bag-of-words descriptors and concatenated to form a single descriptor for the action. The motion pyramidal approach performs better than the baseline improved trajectory descriptors. The method achieves 96.7% on the JPL interaction dataset and 61.8% on our NUS interaction dataset. The same is used to detect actions in long video sequences and achieves average precision of 0.79 on JPL interaction dataset.
BibTeX
@inproceedings{narayan2014action,
  author = {Sanath Narayan and Mohan S. Kankanhalli and Kalpathi Ramakrishnan},
  title = {{Action and Interaction Recognition in First-Person Videos}},
  booktitle = {Workshops @ CVPR},
  year = {2014},
  doi = {10.1109/cvprw.2014.82},
}

Loading related content...

Figure