TL;DR
This paper introduces KeyPose, a novel method for category-level 6D object pose estimation that tackles the challenge of estimating object poses without requiring 3D object models. KeyPose achieves state-of-the-art performance by introducing self-adaptive keypoints, a graph-based feature aggregation mechanism, and a voxelization-based NOCS keypoint prediction strategy.
What Problem Does This Paper Address?
Estimating the 6D pose (3D position and 3D orientation) of objects is crucial for many robotics and augmented reality applications. While instance-level methods can accurately estimate poses for known objects, they fail when encountering novel objects. Category-level pose estimation aims to solve this by predicting poses for any object within a given category, even if it's unseen during training. However, this task is challenging due to significant variations in shape and appearance among objects within the same category. Existing category-level methods often rely on prior 3D models or complex deformation strategies, which can lead to errors. This paper addresses the need for a robust and accurate category-level pose estimation method that doesn't require 3D models and can effectively capture object geometry despite intra-category variations.
Methods
KeyPose introduces several novel components:
- Self-Adaptive Keypoints Detection Module: This module uses a transformer-based approach to predict keypoints that adapt to the specific object's geometry. It fuses RGB and point cloud features, then employs self-attention mechanisms to learn initial keypoint features. A graph module further refines these features by considering geometric relationships.
- Keypoint Graph Feature Aggregation: To capture the geometric relationships between keypoints and the object's point cloud, a graph structure is constructed. This involves finding nearest neighbors in the point cloud for each keypoint and using cosine similarity to adjust keypoint features. Additionally, a keypoint graph is built using DGCNN to extract graph-specific features, which are then fused with the existing keypoint features.
- NOCS Voxelization for Keypoint Prediction: To improve the accuracy of predicting keypoints in Normalized Object Coordinate Space (NOCS), a voxelization strategy is employed. The NOCS space (a 1x1x1 cube) is partitioned into voxels. Instead of directly predicting NOCS keypoint coordinates, the method predicts a heatmap indicating the likelihood of a keypoint being in each voxel. The center of the most likely voxel is then used as the predicted NOCS keypoint location, mitigating positional inaccuracies.
- Pose Estimation: Finally, the estimated geometric keypoints, NOCS keypoints, and their features are fed into an MLP to predict the final 6D object pose (rotation, translation, and scale).
Datasets
The paper evaluates KeyPose on the following datasets:
- CAMERA25: A benchmark dataset for category-level object pose estimation, created by integrating synthetic objects into real-world scenes.
- REAL275: Another benchmark dataset consisting of real-world images from various scenes.
- HouseCat6D: A comprehensive real-world dataset featuring 194 high-fidelity 3D models of household items across 10 categories, including transparent and reflective objects.
Key Insights and Ideas
- Self-adaptive keypoints are crucial: The paper highlights that keypoints that can adapt to object geometry, rather than being fixed, are essential for handling intra-category shape variations.
- Graph structures enhance geometric understanding: By explicitly modeling relationships between keypoints and point clouds, the network gains a richer understanding of object structure, leading to improved pose estimation.
- Voxelization for NOCS prediction is more robust: Directly predicting NOCS keypoints can be prone to errors. The proposed voxel-based approach, by predicting voxel occupancy, offers a more stable and accurate way to localize keypoints in NOCS.
- Combining geometric and NOCS keypoints is powerful: The method effectively leverages both geometric keypoints (in object space) and NOCS keypoints (in a normalized space) to achieve accurate pose estimation.
Key Results
KeyPose demonstrates state-of-the-art performance on benchmark datasets:
- On the CAMERA25 dataset, KeyPose outperforms existing methods by approximately 2%.
- On the REAL275 dataset, KeyPose shows significant improvements, achieving around a 6% gain in metrics like 5°5cm compared to methods like VI-Net and AG-Pose.
- On the HouseCat6D dataset, KeyPose achieves nearly 100% accuracy for transparent objects and shows a notable 20% enhancement in pose accuracy for items like cutlery and shoes at the IoU50 metric.
- Qualitative results show KeyPose producing more accurate pose estimations compared to VI-Net and AG-Pose, especially in cluttered scenes.
Limitations and Future Work
- Ablation studies on loss functions: The paper notes that removing the pose loss function significantly degrades performance, emphasizing its importance. It also shows that removing the keypoint distance loss (La) leads to local clustering, and removing the ground truth alignment loss (LD) causes keypoints to deviate from the object surface. The NOCS keypoint loss (LkNOCS) is also crucial for accurate alignment.
- Future work: The paper does not explicitly detail future work directions, but the success of KeyPose suggests potential avenues for further research could include extending the method to more complex object categories, improving efficiency for real-time applications, and exploring its applicability to other 3D vision tasks.
Summary updated