CGAPoseNet+GCAN: A Geometric Clifford Algebra Network for Geometry-Aware Camera Pose Regression

Alberto Pepe, Joan Lasenby, Sven BuchholzWACV 2024

TL;DR

This paper introduces CGAPoseNet+GCAN, a novel deep learning architecture for camera pose regression that leverages Geometric Clifford Algebra (GCA) to achieve geometry-aware predictions. By integrating a Geometric Clifford Algebra Network (GCAN) into an existing framework, the model significantly improves accuracy and reduces parameters, setting a new state-of-the-art on multiple datasets.

What Problem Does This Paper Address?

Estimating a camera's 3D position and orientation (its "pose") from images is a fundamental problem with applications in augmented reality, robotics, and mapping. Traditional methods often struggle with viewpoint changes and occlusions. While deep learning (DL) approaches like PoseNet have shown promise, they typically treat rotation and translation as separate entities in different mathematical spaces, leading to a "geometry-agnostic" approach. This means the networks don't inherently understand the geometric relationships within the scene or the mathematical properties of pose representations. CGAPoseNet addressed this by unifying rotation and translation into a single mathematical object called a "motor" using GCA, but it still lacked true geometric awareness. This paper aims to bridge this gap by introducing a geometry-aware network that leverages the full power of GCA for pose regression.

Methods

The core innovation is the Geometric Clifford Algebra Network (GCAN), integrated into the CGAPoseNet architecture. Instead of directly predicting a single camera pose (represented as a motor), the backbone network now outputs multiple "motor proposals" (256 in this case). These proposals are then fed into the GCAN, which is designed to operate entirely within the 1D-Up Conformal Geometric Algebra (G4,0) space.

The GCAN consists of a series of sandwich product layers. These layers are crucial because they perform geometric transformations (rotations and translations) in a way that preserves the "grade" of the geometric objects (e.g., a bivector remains a bivector). This ensures that the network's operations are geometrically meaningful. The GCAN progressively downsamples the motor proposals, learning to identify the most accurate pose. This geometry-aware approach allows the network to understand the scene's structure and the inherent properties of poses, leading to more robust and interpretable predictions.

Datasets

The authors evaluated their approach on a combination of indoor and outdoor datasets:

  • 7 Scenes Dataset (Indoor): Includes Chess, Fire, Heads, Office, Pumpkin, Red Kitchen, and Stairs. These scenes are relatively small in volume (< 20m³).
  • Cambridge Landmarks Dataset (Outdoor): Includes Great Court, King's College, Old Hospital, Street, Shop Facade, and St. Mary's Church. These datasets vary significantly in scale and similarity between training and testing data.

These datasets are standard benchmarks for camera pose regression tasks.

Key Insights and Ideas

  • Geometry-Awareness is Key: The paper strongly argues that explicitly incorporating geometric principles (via GCA) into deep learning models for pose estimation leads to significant improvements in accuracy and generalization.
  • GCAN as a Geometric Filter: The GCAN acts as a sophisticated geometric filter, taking multiple plausible pose candidates and refining them into a single, accurate pose by leveraging the inherent geometric structure of the problem.
  • Interpretable Intermediate Representations: Because the GCAN operates with motors, its intermediate layer outputs can be interpreted as poses themselves, offering a degree of explainability not typically found in standard DL models.
  • Unified Representation Power: The use of motors in GCA elegantly handles both rotation and translation within a single mathematical framework, simplifying the loss function and the overall architecture.

Key Results

  • State-of-the-Art Performance: CGAPoseNet+GCAN achieved state-of-the-art results on 11 out of 13 datasets, significantly outperforming previous methods, including CGAPoseNet and the best PoseNet strategies.
  • Reduced Errors: The model achieved a 41% reduction in average rotation error and an 8.8% reduction in average translation error compared to CGAPoseNet. It also outperformed the best PoseNet strategy by 32.6% in rotation error and 19.9% in translation error.
  • Parameter Efficiency: CGAPoseNet+GCAN has approximately 4 million fewer trainable parameters than CGAPoseNet, making it more efficient without sacrificing performance.
  • No Additional Data Required: The approach achieves these improvements without needing extra 3D scene information (like point clouds), relying solely on RGB images.

Limitations and Future Work

  • Backbone Adaptation for Outdoor Scenes: The authors note that for larger outdoor scenes, reshaping the backbone output into more proposals might require GCAN layers with more units to adequately explore the scene's volume and avoid bottlenecks.
  • Activation Function Influence: The choice of activation function in the GCAN can influence the distribution of predicted poses and the convergence behavior, suggesting potential for further optimization.
  • Generalizability of GCANs: The paper suggests that GCANs have broad potential for simplifying and improving various computer vision tasks that have traditionally relied on DL without explicit geometric information. Future work could explore applying GCANs to other geometric deep learning problems.

Summary updated

BibTeX
@inproceedings{pepe2024cgaposenet,
  author = {Alberto Pepe and Joan Lasenby and Sven Buchholz},
  title = {{CGAPoseNet+GCAN: A Geometric Clifford Algebra Network for Geometry-Aware Camera Pose Regression}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure