TL;DR
FPG-NAS is the first differentiable neural architecture search (DNAS) framework tailored specifically for efficient 6DoF object pose estimation. By introducing a task-specific search space, a flexible gating mechanism that allows multiple parallel block selections, and hardware-aware FLOPs regularization, it automatically discovers models that break previous accuracy-efficiency trade-offs.
What Problem Does This Paper Address?
Six Degrees of Freedom (6DoF) pose estimation—predicting an object's 3D position and orientation from a single image—is vital for robotics, augmented reality, and space exploration. However, top-performing models are computationally heavy, making them impractical for edge devices or resource-constrained hardware.
While Neural Architecture Search (NAS) has successfully automated the design of efficient networks for classification and detection, it hasn't been effectively applied to 6DoF pose estimation. Standard DNAS methods rely on a "winner-takes-all" approach (using an argmax step over a softmax distribution) to pick a single operation per layer. The authors argue this is too restrictive for 6DoF tasks, where combining multiple multi-scale features and spatial operations is crucial for capturing complex 3D relationships.
Methods
FPG-NAS introduces a hardware-aware DNAS pipeline optimized for keypoint-based 6DoF pose estimation:
-
Macro & Micro Search Space: The macro architecture consists of 17 backbone layers, a searchable multi-scale fusion layer, and a keypoint heatmap head. The micro-architecture blocks are inspired by EfficientNet and GhostNet, parameterized by kernel sizes and expansion rates.
Fig. 2. Single backbone block. Each candidate selects a kernel size K, an expansion rate e, and a ghost module kernel size Kgh. -
Differentiable Polarized Gates: Instead of standard softmax selection, FPG-NAS uses independent polarized gates: $$g(\alpha) = \frac{\alpha^2}{\alpha^2 + \epsilon}$$ This allows the network to smoothly approximate discrete binary choices while enabling multiple promising candidate blocks to remain active simultaneously within a single layer.
-
FLOPs-Aware Regularization: The supernet loss function includes a specialized penalty term using a ReLU formulation: $$\lambda \cdot \text{ReLU}(\text{FLOPs}_{\text{total}} - \text{FLOPs}_{\text{budget}})$$ This explicitly forces the search algorithm to respect user-defined computational budgets during optimization.
Datasets
The authors evaluate their framework on two prominent benchmarks:
- LINEMOD: A standard dataset containing 16,000 images across 13 object categories, featuring heavily cluttered and challenging real-world scenes.
- SPEED+: A next-generation spacecraft pose estimation dataset containing 60,000 synthetic images of the Tango spacecraft. It is evaluated across specialized test subsets like Lightbox and Sunlamp to simulate harsh, high-contrast space environments.
Key Insights and Ideas
- Synergy Over Selection: Traditional NAS discards runner-up blocks. FPG-NAS shows that allowing multiple complementary blocks to co-exist in a single layer provides the multi-scale spatial reasoning necessary for precise 3D keypoint localization.
- Searchable Fusion Matters: Instead of hardcoding feature pyramid networks (FPN) or simple concatenations, letting the search algorithm choose the cross-scale interaction strategy (e.g., AttentionFusion vs. SEFusion) yields significant efficiency gains.
Key Results
FPG-NAS discovered two primary models, Ours_A (optimized for a 10 GFLOP budget) and Ours_B (optimized for a 5 GFLOP budget).
- LINEMOD:
Ours_Aachieved the highest average accuracy (90.69% ADD(-S)) among all comparable baselines while consuming only 10.03 GFLOPs—outperforming manually designed networks like ADLP (90.40% at 17.3 GFLOPs) and PVNet (86.27% at 72.7 GFLOPs).Ours_Bmaintained a strong 85.19% accuracy at a tiny footprint of 4.37 GFLOPs. - SPEED+:
Ours_Adelivered the lowest computational cost (10.0G) while achieving the best overall accuracy on the Synthetic and Lightbox subsets, outperforming specialized baselines like SPNv2.
Limitations and Future Work
The authors note that the multi-candidate selection mechanism can occasionally activate parallel blocks with highly overlapping characteristics, creating minor redundancies. Future work will explore block-merging strategies to streamline these architectures further. Additionally, they plan to extend FPG-NAS beyond keypoint-heatmap pipelines to direct pose regression and dense correspondence-based 6DoF estimation methods.
Summary updated