HOI-M: Capture Multiple Humans and Objects Interaction within Contextual Environment

Juze Zhang, Jingyan Zhang, Zining Song, Zhanhe Shi, Chengfeng Zhao, Ye Shi, Jingyi Yu, Lan Xu, Jingya WangCVPR 2024 · OpenReview

Abstract

Humans naturally interact with both others and the sur-rounding multiple objects, engaging in various social activi-ties. However, recent advances in modeling human-object interactions mostly focus on perceiving isolated individuals and objects, due to fundamental data scarcity. In this pa-per, we introduce HOI-M 3 , a novel large-scale dataset for modeling the interactions of Multiple huMans and Multiple objects. Notably, it provides accurate 3D tracking for both humans and objects from dense RGB and object-mounted IMU inputs, covering 199 sequences and 181M frames of diverse humans and objects under rich activities. With the unique HOI-M 3 dataset, we introduce two novel data-driven tasks with companion strong baselines: monocular capture and unstructured generation of multiple human-object interactions. Extensive experiments demonstrate that our dataset is challenging and worthy of further research about multiple human-object interactions and behavior analysis. Our HOI-M3 dataset, corresponding codes, and pre-trained models will be disseminated to the community for future research, which can be found at https://juzezhang.github.io/HOIM3_ProjectPage/
BibTeX
@inproceedings{zhang2024hoi,
  author = {Juze Zhang and Jingyan Zhang and Zining Song and Zhanhe Shi and Chengfeng Zhao and Ye Shi and Jingyi Yu and Lan Xu and Jingya Wang},
  title = {{HOI-M: Capture Multiple Humans and Objects Interaction within Contextual Environment}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  url = {https://openreview.net/forum?id=VNDleyc3gy},
}

Loading related content...

Figure