Main objects interaction activity recognition in real images

Lin Bai, Kan Li, JianMeng Pei, Shuai JiangNeural Computing and Applications 2016 · OpenReview

Abstract

Automatically describing the image caption is a challenging task in computer vision. The difficulty mainly lies in capturing the interesting objects and recognizing the interaction activity of the interesting objects. In this paper, we introduce “centerpiece interaction,” a complex visual composite, to represent the main objects interaction activity. We propose a centerpiece interaction recognition framework to achieve the detection of interesting objects and the recognition of their interaction activity by regarding them as an integrated task. In our framework, firstly, a graph-based model is proposed to learn the 2.5D spatial co-occurrence context among objects, which strongly facilitates the interesting objects detection. Secondly, we propose a hierarchical model, with the help of 2.5D spatial co-occurrence context obtained, to learn the relational features of the interesting objects in a hierarchy of stages by integrating the features of the interesting objects, which significantly improve the recognition of centerpiece interaction. Experiments on a joint dataset show that our framework outperforms state-of-the-art in spatial co-occurrence context analysis, the interesting objects detection and the centerpiece interaction recognition.
BibTeX
@article{bai2016main,
  author = {Lin Bai and Kan Li and JianMeng Pei and Shuai Jiang},
  title = {{Main objects interaction activity recognition in real images}},
  journal = {Neural Computing and Applications},
  year = {2016},
  url = {https://openreview.net/forum?id=NaLpukDME1},
}

Loading related content...

Figure