I^2R-Net: Intra- and Inter-Human Relation Network for Multi-Person Pose Estimation

Yiwei Ding, Wenjin Deng, Yinglin Zheng, Pengfei Liu, Meihong Wang, Xuan Cheng, Jianmin Bao, Dong Chen, Ming Zeng

Abstract

In this paper, we present the Intra- and Inter-Human Relation Networks (I^2R-Net) for Multi-Person Pose Estimation. It involves two basic modules. First, the Intra-Human Relation Module operates on a single person and aims to capture Intra-Human dependencies. Second, the Inter-Human Relation Module considers the relation between multiple instances and focuses on capturing Inter-Human interactions. The Inter-Human Relation Module can be designed very lightweight by reducing the resolution of feature map, yet learn useful relation information to significantly boost the performance of the Intra-Human Relation Module. Even without bells and whistles, our method can compete or outperform current competition winners. We conduct extensive experiments on COCO, CrowdPose, and OCHuman datasets. The results demonstrate that the proposed model surpasses all the state-of-the-art methods. Concretely, the proposed method achieves 77.4% AP on CrowPose dataset and 67.8% AP on OCHuman dataset respectively, outperforming existing methods by a large margin.

Introduction

Multi-person pose estimation (MPPE) has traditionally followed two paradigms: top-down (detecting a person, then finding joints) and bottom-up (finding all joints, then grouping them). While top-down methods are highly accurate, they often suffer from "tunnel vision." By treating each person as an isolated bounding box, they ignore the surrounding context. In crowded scenes—such as a dance floor or a busy street—limbs often overlap or are occluded. Without considering the spatial relationships between different individuals, models frequently fail to correctly assign joints.

I^2R-Net addresses this gap by explicitly modeling both intra-human (part-to-part) and inter-human (person-to-person) relationships, allowing the model to "see" and learn from the interactions between multiple people.

The I^2R-Net Framework

The authors propose a two-stage architecture designed to capture hierarchical relational information.

Illustration of (a) intra-human relations (part-to-part) and (b) inter-human relations (person-to-person).

1. Intra-Human Relation Module

This stage operates on high-resolution image patches of individual people to capture dependencies between body parts (e.g., the relationship between a shoulder and an elbow). This module is flexible and can be implemented using existing single-person pose estimators like HRNet or TransPose.

2. Inter-Human Relation Module

This novel addition takes feature maps from the first stage, downsamples them to maintain efficiency, and flattens them into "tokens." These tokens from multiple people are fed into a Transformer block. Using self-attention, the model learns to correlate features across different people. For example, if one person's hand is occluded, the model can use the position of a neighboring person's torso to infer the hand's likely location.

The overall architecture of I^2R-Net, featuring sequential Intra- and Inter-Human Relation Modules.

The two stages are connected via a skip-connection, ensuring the final output benefits from both high-resolution local details and global relational context.

Key Insights and Results

  • Lightweight Relations: A significant finding is that the Inter-Human module does not require high resolution to be effective. By downsampling feature maps, the authors added relational intelligence with a negligible increase in parameters (~0.5M).
  • Spatial Importance: Ablation studies showed that the model needs at least a small spatial grid (e.g., 16x12) to understand where parts are relative to each other across different people; reducing this to a single token per person caused a drop in accuracy.
  • State-of-the-Art Performance: I^2R-Net sets a new bar for performance in crowded environments. It achieved 77.4% AP on CrowdPose (a 5.0% improvement over HRFormer-B) and 67.8% AP on OCHuman (a 5.5% improvement over TransPose-H).

Conclusion

I^2R-Net demonstrates that explicitly modeling inter-human interactions is a powerful strategy for overcoming occlusion in multi-person pose estimation. By augmenting standard architectures with a lightweight Transformer-based relational stage, the network achieves superior results in crowded scenes with minimal computational overhead. Future research may extend these relational insights into the temporal domain for video sequences.

Summary updated

Abstract

In this paper, we present the Intra- and Inter-Human Relation Networks (I^2R-Net) for Multi-Person Pose Estimation. It involves two basic modules. First, the Intra-Human Relation Module operates on a single person and aims to capture Intra-Human dependencies. Second, the Inter-Human Relation Module considers the relation between multiple instances and focuses on capturing Inter-Human interactions. The Inter-Human Relation Module can be designed very lightweight by reducing the resolution of feature map, yet learn useful relation information to significantly boost the performance of the Intra-Human Relation Module. Even without bells and whistles, our method can compete or outperform current competition winners. We conduct extensive experiments on COCO, CrowdPose, and OCHuman datasets. The results demonstrate that the proposed model surpasses all the state-of-the-art methods. Concretely, the proposed method achieves 77.4% AP on CrowPose dataset and 67.8% AP on OCHuman dataset respectively, outperforming existing methods by a large margin. Additionally, the ablation study and visualization analysis also prove the effectiveness of our model.
BibTeX
@inproceedings{ding2022i,
  author = {Yiwei Ding and Wenjin Deng and Yinglin Zheng and Pengfei Liu and Meihong Wang and Xuan Cheng and Jianmin Bao and Dong Chen and Ming Zeng},
  title = {{I^2R-Net: Intra- and Inter-Human Relation Network for Multi-Person Pose Estimation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  url = {https://openreview.net/forum?id=54TVnEvoUf},
}
Figure