Abstract
Category-level 6D object pose and size estimation is to predict full pose configurations of rotation, translation, and size for object instances observed in single, arbitrary views of cluttered scenes. In this paper, we propose a new method of Dual Pose Network with refined learning of pose consistency for this task, shortened as DualPoseNet. DualPoseNet stacks two parallel pose decoders on top of a shared pose encoder, where the implicit decoder predicts object poses with a working mechanism different from that of the explicit one; they thus impose complementary supervision on the training of pose encoder. We construct the encoder based on spherical convolutions, and design a module of Spherical Fusion wherein for a better embedding of pose-sensitive features from the appearance and shape observations. Given no testing CAD models, it is the novel introduction of the implicit decoder that enables the refined pose prediction during testing, by enforcing the predicted pose consistency between the two decoders using a self-adaptive loss term. Thorough experiments on benchmarks of both category- and instance-level object pose datasets confirm efficacy of our designs. DualPoseNet outperforms existing methods with a large margin in the regime of high precision.
Introduction
Estimating the 6D pose (3D rotation and 3D translation) and size of an object from a single RGB-D image is a cornerstone of robotics, augmented reality, and autonomous driving. While "instance-level" estimation—where a perfect 3D CAD model of the specific object is available—is well-studied, "category-level" estimation presents significant hurdles. In category-level tasks, the model must generalize to novel instances (e.g., an unseen mug) within a known category. This involves overcoming intra-category variation, the absence of reference models at test time for refinement, and the inherent difficulty standard neural networks face in learning features equivariant to 3D rotations ($SO(3)$).
Methodology
The core of our approach is the DualPoseNet architecture, which leverages a shared encoder and two distinct decoding paths to achieve high-precision pose estimation.
Dual Architecture and Spherical Fusion
The architecture begins with a Pose Encoder that utilizes spherical convolutions to process RGB and Depth data. Unlike standard 2D or 3D convolutions, spherical convolutions are naturally suited for 3D rotations, allowing the network to learn features that are sensitive to orientation. We introduce a Spherical Fusion module to effectively integrate appearance and shape features across multiple scales.
Two parallel decoders are stacked on this encoder:
- Explicit Decoder: Directly regresses the rotation (as a quaternion), translation, and scale of the object.
- Implicit Decoder: Reconstructs the observed point cloud into a "canonical" pose—a standardized orientation for that category. This provides auxiliary supervision during training, forcing the encoder to learn robust geometric features.
Refined Learning of Pose Consistency
A key novelty of DualPoseNet is the ability to refine predictions at test time without a CAD model. By calculating a "consistency loss" between the explicit pose and the implicit reconstruction, the model can perform a rapid optimization step. This fine-tunes the encoder's weights to ensure both decoders agree, effectively correcting errors on the fly.
Experimental Results
DualPoseNet was evaluated on major benchmarks, including CAMERA25 (synthetic) and REAL275 (real-world), as well as instance-level datasets like YCB-Video and LineMOD.
Performance Metrics
The method significantly outperforms previous state-of-the-art approaches, particularly in high-precision regimes. On the REAL275 dataset, at a strict threshold of $5^\circ$ rotation error and $2cm$ translation error, DualPoseNet achieved a 29.3% mAP, nearly tripling the performance of the baseline NOCS method (7.2%).
Impact of Refinement
The "Refined Learning" step consistently improved results. For instance, it boosted the $10^\circ, 10\%$ mAP on REAL275 from 43.1% to 44.5%. This refinement process is efficient, taking approximately 0.2 seconds per instance on a standard GPU.
Conclusion
DualPoseNet demonstrates that duality in pose decoding provides powerful self-supervision and enables test-time refinement without external CAD models. While the system currently depends on initial 2D segmentation (e.g., Mask R-CNN) and faces challenges with extreme symmetry, it sets a new standard for high-precision category-level pose estimation. Future work may explore extending these principles to open-category scenarios.
Summary updated