FPG-NAS: FLOPs-Aware Gated Differentiable Neural Architecture Search for Efficient 6DoF Pose Estimation

Nassim Ali Ousalah, Peyman Rostami, Anis Kacem, Enjie Ghorbel, Emmanuel Koumandakis, Djamila Aouada

TL;DR

FPG-NAS is the first differentiable neural architecture search (DNAS) framework tailored specifically for efficient 6DoF object pose estimation. By introducing a task-specific search space, a flexible gating mechanism that allows multiple parallel block selections, and hardware-aware FLOPs regularization, it automatically discovers models that break previous accuracy-efficiency trade-offs.

What Problem Does This Paper Address?

Six Degrees of Freedom (6DoF) pose estimation—predicting an object's 3D position and orientation from a single image—is vital for robotics, augmented reality, and space exploration. However, top-performing models are computationally heavy, making them impractical for edge devices or resource-constrained hardware.

While Neural Architecture Search (NAS) has successfully automated the design of efficient networks for classification and detection, it hasn't been effectively applied to 6DoF pose estimation. Standard DNAS methods rely on a "winner-takes-all" approach (using an argmax step over a softmax distribution) to pick a single operation per layer. The authors argue this is too restrictive for 6DoF tasks, where combining multiple multi-scale features and spatial operations is crucial for capturing complex 3D relationships.

Methods

FPG-NAS introduces a hardware-aware DNAS pipeline optimized for keypoint-based 6DoF pose estimation:

  • Macro & Micro Search Space: The macro architecture consists of 17 backbone layers, a searchable multi-scale fusion layer, and a keypoint heatmap head. The micro-architecture blocks are inspired by EfficientNet and GhostNet, parameterized by kernel sizes and expansion rates.

    Fig. 2. Single backbone block. Each candidate selects a kernel size K, an expansion rate e, and a ghost module kernel size Kgh.
  • Differentiable Polarized Gates: Instead of standard softmax selection, FPG-NAS uses independent polarized gates: $$g(\alpha) = \frac{\alpha^2}{\alpha^2 + \epsilon}$$ This allows the network to smoothly approximate discrete binary choices while enabling multiple promising candidate blocks to remain active simultaneously within a single layer.

  • FLOPs-Aware Regularization: The supernet loss function includes a specialized penalty term using a ReLU formulation: $$\lambda \cdot \text{ReLU}(\text{FLOPs}_{\text{total}} - \text{FLOPs}_{\text{budget}})$$ This explicitly forces the search algorithm to respect user-defined computational budgets during optimization.

Datasets

The authors evaluate their framework on two prominent benchmarks:

  • LINEMOD: A standard dataset containing 16,000 images across 13 object categories, featuring heavily cluttered and challenging real-world scenes.
  • SPEED+: A next-generation spacecraft pose estimation dataset containing 60,000 synthetic images of the Tango spacecraft. It is evaluated across specialized test subsets like Lightbox and Sunlamp to simulate harsh, high-contrast space environments.

Key Insights and Ideas

  • Synergy Over Selection: Traditional NAS discards runner-up blocks. FPG-NAS shows that allowing multiple complementary blocks to co-exist in a single layer provides the multi-scale spatial reasoning necessary for precise 3D keypoint localization.
  • Searchable Fusion Matters: Instead of hardcoding feature pyramid networks (FPN) or simple concatenations, letting the search algorithm choose the cross-scale interaction strategy (e.g., AttentionFusion vs. SEFusion) yields significant efficiency gains.

Key Results

FPG-NAS discovered two primary models, Ours_A (optimized for a 10 GFLOP budget) and Ours_B (optimized for a 5 GFLOP budget).

Architectures discovered under different FLOP constraints
  • LINEMOD: Ours_A achieved the highest average accuracy (90.69% ADD(-S)) among all comparable baselines while consuming only 10.03 GFLOPs—outperforming manually designed networks like ADLP (90.40% at 17.3 GFLOPs) and PVNet (86.27% at 72.7 GFLOPs). Ours_B maintained a strong 85.19% accuracy at a tiny footprint of 4.37 GFLOPs.
  • SPEED+: Ours_A delivered the lowest computational cost (10.0G) while achieving the best overall accuracy on the Synthetic and Lightbox subsets, outperforming specialized baselines like SPNv2.

Limitations and Future Work

The authors note that the multi-candidate selection mechanism can occasionally activate parallel blocks with highly overlapping characteristics, creating minor redundancies. Future work will explore block-merging strategies to streamline these architectures further. Additionally, they plan to extend FPG-NAS beyond keypoint-heatmap pipelines to direct pose regression and dense correspondence-based 6DoF estimation methods.

Summary updated

Abstract

We introduce FPG-NAS, a FLOPs-aware Gated Differentiable Neural Architecture Search framework for efficient 6DoF object pose estimation. Estimating 3D rotation and translation from a single image has been widely investigated yet remains computationally demanding, limiting applicability in resource-constrained scenarios. FPG-NAS addresses this by proposing a specialized differentiable NAS approach for 6DoF pose estimation, featuring a task-specific search space and a differentiable gating mechanism that enables discrete multi-candidate operator selection, thus improving architectural diversity. Additionally, a FLOPs regularization term ensures a balanced trade-off between accuracy and efficiency. The framework explores a vast search space of approximately 10\textsuperscript{92} possible architectures. Experiments on the LINEMOD and SPEED+ datasets demonstrate that FPG-NAS-derived models outperform previous methods under strict FLOPs constraints. To the best of our knowledge, FPG-NAS is the first differentiable NAS framework specifically designed for 6DoF object pose estimation.
BibTeX
@misc{ousalah2025fpg,
  author = {Nassim Ali Ousalah and Peyman Rostami and Anis Kacem and Enjie Ghorbel and Emmanuel Koumandakis and Djamila Aouada},
  title = {{FPG-NAS: FLOPs-Aware Gated Differentiable Neural Architecture Search for Efficient 6DoF Pose Estimation}},
  year = {2025},
  eprint = {2508.03618},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.03618},
}
Figure