TL;DR
This paper introduces COPE, a novel end-to-end trainable framework for estimating the 6D pose of multiple objects in a single image. COPE achieves near-constant runtime regardless of the number of object instances, significantly outperforming previous methods in speed while maintaining competitive accuracy.
What Problem Does This Paper Address?
Estimating the 6D pose (3D position and 3D orientation) of objects from a single image is a fundamental problem in computer vision with applications in robotics, augmented reality, and autonomous driving. A major challenge arises when dealing with multiple instances of objects within the same image. Traditional approaches often involve a two-stage process: first detecting objects and then estimating the pose for each detected object individually. This multi-model approach is computationally expensive and scales poorly with an increasing number of objects. Existing methods that attempt to address this by directly regressing 6D poses often struggle with efficient multi-instance handling or require separate object detectors, hindering end-to-end trainability.
Methods
COPE tackles these limitations by proposing an end-to-end trainable, single-stage framework. The core idea is to share a latent representation and a direct pose regressor across all objects and their instances. The network takes an RGB image as input and simultaneously predicts:
- Object Class Probabilities: Identifying the class of each potential object.
- Amodal Bounding Boxes: Predicting bounding boxes that encompass the entire object, even occluded parts.
- Geometric Correspondences: Estimating 2D projections of 3D keypoints of the object's bounding cuboid.
- Direct 6D Pose: Directly regressing the 6D pose (rotation and translation) for each object instance.
A key innovation is the use of a geometry-based approach for sampling training locations and standardizing regression targets that accounts for object shape and scale, reducing the need for manual hyperparameter tuning compared to anchor-based methods. The framework also incorporates a symmetry-aware loss to handle objects with rotational or reflectional symmetries, which can otherwise confuse pose estimation. During testing, COPE efficiently clusters pose hypotheses using the Intersection-over-Union (IoU) of predicted bounding boxes and then averages the top-consistent hypotheses to produce the final pose.
Datasets
The paper evaluates COPE on three standard datasets:
- LM (Linemod): A common benchmark for object pose estimation in cluttered environments.
- LM-O (Linemod Occlusion): A more challenging variant of LM with significant object occlusions.
- IC-BIN: A dataset featuring multiple instances of objects with heavy occlusion.
Additionally, a synthetic dataset (IC-BIN syn) was generated using OpenGL2 rendering to specifically test the method's scalability with up to 100 object instances per image.
Key Insights and Ideas
- End-to-End Trainability for Multi-Instance Pose: By sharing representations and directly regressing poses, COPE achieves end-to-end training, which is crucial for optimizing the entire pipeline and improving performance.
- Constant Runtime is Achievable: The paper demonstrates that it's possible to process a large number of object instances (up to 70 with negligible runtime increase, and >90 at real-time speeds) in a single forward pass, a significant improvement over previous methods.
- Geometry-Informed Training: Using object shape and scale to guide training target sampling and standardization leads to more robust and efficient learning, reducing reliance on hand-tuned parameters.
- Direct Pose Regression Benefits Geometric Correspondences: The proposed loss formulation, which supervises geometric correspondence learning with direct pose regression, leads to improved intermediate representations, especially in challenging scenarios like occlusion.
Key Results
- Speed: COPE is up to 35 times faster than comparable state-of-the-art methods, achieving near-constant runtime even with many object instances. It can process over 90 instances per image at >24 fps.
- Accuracy: COPE achieves competitive pose estimation accuracy, outperforming previous single-model approaches and being comparable to multi-model methods, despite its significant speed advantage. For instance, on IC-BIN, COPE achieves an Average Recall (AR) of 0.440, compared to EPOS's 0.455 (single-model) and CosyPose's 0.596 (multi-model).
- Object Detection: COPE also demonstrates strong object detection capabilities, achieving the highest average mAP on IC-BIN and LM-O compared to MaskRCNN and FCOS.
Limitations and Future Work
- Geometric Correspondences as a Dependency: While COPE learns intermediate geometric correspondences, the authors acknowledge that defining these correspondences is still a manual step.
- Future Work: The authors plan to explore self-supervised learning for the intermediate representation to eliminate the need for explicitly defining geometric correspondences. This would further enhance the end-to-end nature of the framework.
Summary updated