Abstract
This paper focuses on the regression of multiple 3D people from a single RGB image. Existing approaches predominantly follow a multi-stage pipeline that first detects people in bounding boxes and then independently regresses their 3D body meshes. In contrast, we propose to Regress all meshes in a One-stage fashion for Multiple 3D People (termed ROMP). The approach is conceptually simple, bounding box-free, and able to learn a per-pixel representation in an end-to-end manner. Our method simultaneously predicts a Body Center heatmap and a Mesh Parameter map, which can jointly describe the 3D body mesh on the pixel level. Through a body-center-guided sampling process, the body mesh parameters of all people in the image are easily extracted from the Mesh Parameter map. Equipped with such a fine-grained representation, our one-stage framework is free of the complex multi-stage process and more robust to occlusion. Compared with state-of-the-art methods, ROMP achieves superior performance on the challenging multi-person benchmarks, including 3DPW and CMU Panoptic. Experiments on crowded/occluded datasets demonstrate the robustness under various types of occlusion.
Introduction
Estimating the 3D shape and pose of humans from a single image is a significant challenge, particularly in crowded scenes. Traditional "top-down" methods rely on a multi-stage pipeline: first detecting individuals within bounding boxes and then independently regressing 3D meshes for each crop. This approach is computationally expensive due to redundant feature extraction and often fails in "in-the-wild" scenarios involving truncation or heavy person-to-person occlusion. When individuals overlap significantly, their bounding boxes become nearly identical, leading to feature ambiguity and "center collision," where the model fails to distinguish between different people.
Methodology: The ROMP Framework
ROMP (Regress all meshes in a One-stage fashion for Multiple 3D People) reformulates 3D mesh recovery as a dense, pixel-level prediction task, eliminating the need for bounding boxes.
One-Stage Architecture
ROMP utilizes a backbone (e.g., ResNet-50 or HRNet-32) to simultaneously predict two primary maps:
- Body Center Heatmap ($C_m$): A probability map where peaks indicate the centers of detected individuals.
- Mesh Parameter Map ($P_m$): A dense map containing camera and SMPL (Skinned Multi-Person Linear model) parameters for every pixel.
Parameter Sampling and Collision Awareness
Final 3D meshes are obtained by sampling parameters from the Mesh Parameter Map at the peak locations identified in the Body Center Heatmap. A key innovation is the Collision-Aware Representation (CAR). By treating body centers like positive charges in a repulsion field during training, the model is forced to learn features from visible body parts, effectively disambiguating overlapping individuals and resolving the "center collision" problem.
Experimental Results
ROMP was evaluated on a wide range of 3D (Human3.6M, MPI-INF-3DHP) and 2D (MS COCO, Crowdpose) datasets.
- Accuracy: ROMP achieved state-of-the-art results on the 3DPW and CMU Panoptic benchmarks. On 3DPW, it significantly reduced the Mean Per Joint Position Error (MPJPE) to 80.1 mm, compared to 93.5 mm for previous leaders like VIBE.
- Efficiency: As a one-stage model, ROMP achieves real-time performance, running at over 30 FPS on an NVIDIA 1070Ti GPU, making it the first real-time implementation of monocular multi-person 3D mesh regression.
- Robustness: The CAR module provided a 10% performance boost on the Crowdpose dataset, demonstrating its efficacy in dense, occluded environments.
Conclusion and Future Directions
ROMP represents a shift toward holistic, contextual awareness in 3D human pose estimation. By ditching bounding boxes for a pixel-level representation, it simplifies the architecture and improves robustness. Future work will focus on addressing extreme depth complexity (e.g., people stacked directly behind one another), refining depth estimation beyond the "weak-perspective" model, and explicitly modeling human-to-human interactions.
Summary updated