Detecting Human-Object Relationships in Videos

Jingwei Ji, Rishi Desai, Juan Carlos NieblesICCV 2021 · OpenReview

Abstract

We study a crucial problem in video analysis: human-object relationship detection. The majority of previous approaches are developed only for the static image scenario, without incorporating the temporal dynamics so vital to contextualizing human-object relationships. We propose a model with Intra- and Inter-Transformers, enabling joint spatial and temporal reasoning on multiple visual concepts of objects, relationships, and human poses. We find that applying attention mechanisms among features distributed spatio-temporally greatly improves our understanding of human-object relationships. Our method is validated on two datasets, Action Genome and CAD-120-EVAR, and achieves state-of-the-art performance on both of them.
BibTeX
@inproceedings{ji2021detecting,
  author = {Jingwei Ji and Rishi Desai and Juan Carlos Niebles},
  title = {{Detecting Human-Object Relationships in Videos}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=xeEY8MPtrC},
}

Loading related content...

Figure