GFA-Net: Geometry-Focused Attention Network for Six Degrees of Freedom Object Pose Estimation

Shuai Lin, Junhui Yu, Peng Su, Weitao Xue, Qiang Yang, Lina Fu, Jing Wen, Hong HuangSensors 2024 · DOI

TL;DR

This paper introduces GFA-Net, a novel deep learning framework for estimating the 6-DoF pose of objects from single RGB images. By focusing on extracting rich geometric and textural features using novel attention mechanisms, GFA-Net achieves state-of-the-art performance on challenging benchmarks, even in the presence of occlusions, without relying on depth data.

What Problem Does This Paper Address?

Estimating the precise 6-DoF (position and orientation) of an object from a single 2D RGB image is a fundamental problem in robotics and computer vision, crucial for tasks like robotic grasping and autonomous driving. While many methods exist, a significant challenge is accurately extracting detailed geometric information from RGB images alone, especially when objects have limited texture or are partially occluded. Existing approaches often resort to using supplementary data like depth maps or 3D models, which are not always available. Furthermore, deep neural networks can struggle to capture subtle geometric cues from RGB images, leading to pose estimation errors. This paper aims to bridge this gap by developing a purely RGB-based method that excels at geometric feature extraction.

Methods

The proposed GFA-Net is a multi-stage framework:

  • Object Detector: An initial object detector (YOLOv3 in their implementation) localizes the object of interest within the RGB image, providing a bounding box.
  • Geometric Feature Extractor: This core component is responsible for extracting rich geometric and textural information. It consists of:
  • Feature Extractor: A High-Resolution Network (HRNet) is used as a backbone to maintain multi-scale feature representations.
  • Point-wise Feature Attention (PFA): This novel module captures subtle pose changes by focusing on pixel-level feature variations. It encodes spatial information by decomposing 2D pooling into two 1D operations and uses attention weights to emphasize important pixel features, enhancing spatial relationships.
  • Geometry Feature Aggregation Module (GFAM): This module aggregates multi-scale feature maps generated by PFA. It uses a combination of convolutional layers and attention mechanisms to fuse features and identify long-range dependencies, distilling crucial geometric information.
  • Dense Correspondence Generation: The features extracted by GFAM are used to generate dense 2D-3D correspondences, mapping each pixel of the object to its 3D world coordinate. This is achieved by predicting a geometric coordinate map (MGC) and geometry attention weights (WGA).
  • PnP Module: Finally, a Perspective-n-Point (PnP) solver, specifically EPro-PnP, uses the generated dense correspondences and their associated weights to compute the final 6-DoF object pose.

Datasets

The paper evaluates GFA-Net on two standard benchmarks:

  • LINEMOD Dataset: A widely used benchmark for 6-DoF object pose estimation, featuring 13 object instances with varying geometric and textural properties, including cluttered scenes and low-texture objects.
  • Occlusion LINEMOD Dataset: A subset of LINEMOD that specifically introduces significant occlusion, making pose estimation more challenging.

The authors also mention using synthetic data for training, including 1000 synthetic images per object for LINEMOD and 10,000 rendered synthetic images for the Occlusion LINEMOD dataset, with backgrounds from PASCAL VOC2012.

Key Insights and Ideas

  • RGB-only is Sufficient: The research demonstrates that high-accuracy 6-DoF pose estimation can be achieved using only RGB images, without the need for depth sensors or explicit 3D models during inference.
  • Geometry is Key: The paper strongly emphasizes the importance of explicitly extracting and aggregating geometric features from RGB images, arguing that this is crucial for robust pose estimation, especially in challenging scenarios.
  • Pixel-Level Attention for Pose: The Point-wise Feature Attention (PFA) module is a novel contribution that effectively captures subtle pose shifts by focusing on pixel-level changes and spatial relationships, which is critical for fine-grained pose accuracy.
  • Multi-Scale Feature Fusion: The Geometry Feature Aggregation Module (GFAM) highlights the benefit of combining features from different scales to capture comprehensive geometric information and long-range dependencies.
  • Attention for Dense Correspondences: The generated attention weights (WGA) play a vital role in guiding the PnP solver, improving the efficiency and accuracy of using dense correspondences.

Key Results

GFA-Net demonstrates highly competitive performance on challenging datasets:

  • LINEMOD Dataset: Achieved an accuracy of 96.54% using the ADD-S metric with a 0.10d threshold.
  • Occlusion LINEMOD Dataset: Achieved an accuracy of 49.35% using the ADD-S metric with a 0.10d threshold, showing robustness to occlusion.
  • Ablation studies show that both PFA and GFAM modules contribute significantly to the overall performance, with their combined use leading to substantial improvements, especially under stricter ADD metric thresholds (e.g., 0.05d and 0.02d).

Limitations and Future Work

  • Limitations: While GFA-Net performs well, the paper acknowledges that performance can still be affected by severe occlusions and segmentation errors, as noted in the discussion of the CSA6D method. The paper also mentions that coordinate-based methods can sometimes lead to efficiency issues due to the density of correspondences.
  • Future Work: The authors plan to focus on leveraging attention mechanisms for more efficient geometric feature extraction and developing more general and efficient geometric feature extraction modules.

Summary updated

Abstract

Six degrees of freedom (6-DoF) object pose estimation is essential for robotic grasping and autonomous driving. While estimating pose from a single RGB image is highly desirable for real-world applications, it presents significant challenges. Many approaches incorporate supplementary information, such as depth data, to derive valuable geometric characteristics. However, the challenge of deep neural networks inadequately extracting features from object regions in RGB images remains. To overcome these limitations, we introduce the Geometry-Focused Attention Network (GFA-Net), a novel framework designed for more comprehensive feature extraction by analyzing critical geometric and textural object characteristics. GFA-Net leverages Point-wise Feature Attention (PFA) to capture subtle pose differences, guiding the network to localize object regions and identify point-wise discrepancies as pose shifts. In addition, a Geometry Feature Aggregation Module (GFAM) integrates multi-scale geometric feature maps to distill crucial geometric features. Then, the resulting dense 2D-3D correspondences are passed to a Perspective-n-Point (PnP) module for 6-DoF pose computation. Experimental results on the LINEMOD and Occlusion LINEMOD datasets indicate that our proposed method is highly competitive with state-of-the-art approaches, achieving 96.54% and 49.35% accuracy, respectively, utilizing the ADD-S metric with a 0.10d threshold.
BibTeX
@inproceedings{lin2024gfa,
  author = {Shuai Lin and Junhui Yu and Peng Su and Weitao Xue and Qiang Yang and Lina Fu and Jing Wen and Hong Huang},
  title = {{GFA-Net: Geometry-Focused Attention Network for Six Degrees of Freedom Object Pose Estimation}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s25010168},
}
Figure