Abstract
Estimating the 6-Degree-of-Freedom (6DoF) pose—rotation, translation, and size—of objects at the category level is a critical task for robotics and augmented reality. Unlike instance-level estimation, category-level tasks must account for significant intra-category shape variations. Traditional methods address this by deforming a dense, pre-calculated "mean shape" point cloud to match observed data. We introduce Query6DoF, a framework that replaces these redundant, static priors with a set of sparse, learnable queries. By optimizing these implicit embeddings alongside the pose estimation task, Query6DoF achieves state-of-the-art accuracy on the NOCS benchmark, reaching 49.0% on the strict $5^\circ 2cm$ metric at 34.9 FPS.
Introduction: Rethinking Shape Priors
The standard paradigm for category-level pose estimation involves a "shape prior"—a generic 3D model representing a category (e.g., a "mean" bottle). Networks typically attempt to deform this prior to align with the observed point cloud of a specific instance. However, this approach suffers from computational inefficiency due to dense point clouds (often 1,024 points), static priors that are not optimized for the end task, and a reliance on surrogate reconstruction objectives.
Query6DoF shifts this paradigm by moving the "deformation-and-matching" process into a latent feature space using sparse, learnable embeddings called queries.
Methodology
The Query6DoF architecture is designed to be both efficient and highly accurate by focusing on semantic landmarks rather than dense geometry.
Sparse Learnable Queries
Instead of a thousand-point cloud, Query6DoF uses a small set of learnable vectors (queries). Each query is intended to capture a specific semantic component of a category, such as the handle of a mug or the screen of a laptop. These queries are optimized via backpropagation, allowing the network to "learn" the most effective implicit representation for pose estimation.
Network Architecture
The pipeline consists of three main stages:
- Queries Deformation Estimator: Utilizing cross-attention, the category-level queries "attend" to the features of the observed object point cloud ($P_{obj}$), transforming them into instance-specific deformed queries.
- Query-Point Correspondence Estimator: This module establishes a relationship between the deformed queries and the input points to generate a normalized coordinate space (NOCS) representation.
- Poses and Sizes Estimator: Rather than using traditional geometric fitting algorithms like Umeyama, Query6DoF directly regresses the rotation ($R$), translation ($t$), and size ($s$) using parallel Multi-Layer Perceptrons (MLPs).
Global Enhancement Block (GEB)
To maintain real-time performance, the authors introduced the Global Enhancement Block. Standard self-attention mechanisms have quadratic complexity $O(N^2)$, which is prohibitive for dense point clouds. The GEB uses global average pooling to inject global context into local features with linear complexity $O(N)$, ensuring the model remains lightweight without sacrificing contextual awareness.
Experimental Results
The model was evaluated on the NOCS-REAL275 and CAMERA25 datasets. Query6DoF demonstrated a significant performance leap over previous state-of-the-art methods:
- High Precision: On the challenging $5^\circ 2cm$ metric, Query6DoF achieved 49.0% accuracy, outperforming the previous leader, SGPA (35.9%), by a wide margin.
- Efficiency: The model achieves an inference speed of 34.9 FPS on an RTX 2080Ti, making it suitable for real-time robotic manipulation and AR applications.
- Ablation Study: Experiments confirmed that the use of sparse queries is the primary driver of performance. Replacing queries with a standard point cloud prior led to a substantial drop in accuracy, validating the "implicit shape prior" concept.
Conclusion
Query6DoF demonstrates that sparse, learnable queries are a more effective and efficient way to represent category-level shape priors than traditional dense point clouds. By optimizing these queries directly for the pose estimation task and utilizing an attention-based architecture, the model sets a new standard for both accuracy and speed in the field. Future work may involve tighter integration with object detection and extending the framework to articulated objects.
Summary updated