Abstract
This paper focuses on multi-person 3D mesh recovery from a single RGB image. Existing approaches predominantly follow a multi-stage pipeline that detects bounding boxes and then regresses the body from bounding-box-level features. However, multi-person occlusion and truncation can make these features ambiguous, resulting in recovery failure. To address this, we present a novel bottom-up single-shot method named Center-based Human Mesh Recovery network (CenterHMR), also known as ROMP. The key idea is an explicit center-based representation for bottom-up pixel-level estimation. Guided by body centers, our model effectively locates every person and learns robust features under occlusion. Furthermore, a collision-aware center representation ensures distinguishable distances between body centers in crowded scenes. CenterHMR achieves state-of-the-art performance on four challenging benchmarks and is the first open-source real-time (over 30 FPS) implementation of monocular multi-person 3D mesh recovery.
Introduction: The Challenge of Multi-Person Occlusion
Most 3D human mesh recovery (HMR) methods follow a "top-down" approach: first, detect people using bounding boxes, then crop those boxes and regress the 3D mesh for each person individually. While effective for single-person images, this strategy often fails in the real world. In crowded scenes, bounding boxes overlap significantly, making the features inside a box ambiguous—it becomes unclear which limbs belong to which person. Furthermore, when people are truncated by the image frame, detection often fails entirely.
The authors argue that a "bottom-up" approach—one that looks at the whole scene holistically rather than through isolated crops—is necessary for robust, multi-person 3D estimation.
Methodology: The ROMP Framework
The proposed framework, ROMP (Regress all meshes in a One-stage fashion for Multiple 3D People), predicts three dense, differentiable maps from the entire image:
- Body Center Heatmap ($C_m$): Predicts the probability of a pixel being the center of a human body (specifically, the average of visible torso joints).
- Camera Map ($A_m$): Predicts camera parameters (scale and translation) for a person centered at that pixel.
- SMPL Map ($S_m$): Predicts the 142 parameters required for the SMPL body model (pose and shape) for a person at that pixel.
Collision-Aware Representation (CAR)
A major challenge for bottom-up methods is "center collision"-when two people overlap so much that their centers occupy the same pixel. To solve this, the authors treat body centers like like-charged particles. During training, if two centers are too close, a repulsion field pushes them apart. This forces the model to learn to "sample" mesh parameters from visible body parts rather than a single ambiguous point, significantly improving robustness in crowds.
Experiments and Results
The model was trained on a massive suite of data, including 3D datasets (Human3.6M, MPI-INF-3DHP), pseudo-3D data (UP), and 2D pose datasets (MS COCO, MPII, LSP, AICH). Evaluation focused on challenging benchmarks like 3DPW (in-the-wild), CMU Panoptic (multi-person), and Crowdpose.
- State-of-the-Art Accuracy: On the 3DPW dataset, ROMP achieved an MPJPE (Mean Per Joint Position Error) of 80.1 mm, significantly outperforming previous multi-stage SOTA methods like VIBE (93.5 mm).
- Crowd Robustness: On the CMU Panoptic benchmark, ROMP reduced the error by over 10% compared to the previous best multi-person method (CRMH).
- Real-Time Speed: Because it is a one-stage, anchor-free model, it avoids the computational overhead of region proposals. ROMP runs at roughly 30.9 FPS on a standard NVIDIA 1070Ti GPU, with processing time remaining nearly constant regardless of the number of people in the scene.
Key Insights and Future Work
By looking at the whole image, the model learns spatial relationships between people, allowing it to disambiguate which limbs belong to whom. The CAR module provides a clever way to handle the inherent ambiguity of 2D projections by forcing the model to distinguish between overlapping centers.
While ROMP sets a new standard for real-time multi-person HMR, limitations remain. Complex depth relationships, such as people hugging or intertwining, still pose a challenge for perfect rendering. Future directions include integrating more fine-grained pixel-level depth estimation directly into the one-stage framework to further improve performance in extremely crowded or complex environments.
Summary updated