Pre-defined Keypoints Promote Category-level Articulation Pose Estimation via Multi-Modal Alignment

Wenbo Xu, Li Zhang, Liu Liu, Yan Zhong, Haonan Jiang, Xue Wang, Rujing WangIJCAI 2025 · DOI

Abstract

Estimating the 6D pose of articulated objects—items with moving parts connected by joints—is a fundamental challenge in robotic perception. Traditional methods often struggle with uneven keypoint distributions and inefficient multi-modal data fusion. We present PAGE (Pre-defined keypoint based framework for category-level articulation pose estimation via multi-modal AliGnmEnt), a novel framework that utilizes a "pre-defined" keypoint strategy and geometry-guided fusion of RGB and Depth data. PAGE achieves state-of-the-art accuracy across synthetic and real-world datasets, significantly reducing rotation and translation errors in complex articulated structures.

Introduction

Articulated object pose estimation is significantly more complex than rigid object tracking due to the high degrees of freedom involved. Current approaches typically rely on heuristic keypoints, such as those selected via Farthest Point Sampling (FPS) or bounding box corners. However, these heuristics often result in uneven distance distributions relative to the object's visible surface, creating a "divergent distance pattern" that biases neural network learning. Furthermore, standard multi-modal fusion techniques—like simple concatenation of RGB and point cloud data—often fail to handle information redundancy or noise mismatches effectively.

Conceptual comparison between prior heuristic-based methods and the proposed pre-defined keypoint scoring approach.

Methodology: The PAGE Framework

The PAGE framework introduces three core components designed to stabilize geometric learning and improve feature integration:

1. Pre-defined Keypoint Estimation (KP-Estimator)

Instead of arbitrary heuristics, PAGE employs a Graph Network to learn a set of "pre-defined" keypoints. These points are optimized using a Wasserstein loss to ensure a uniform radius distribution relative to the object's surface. This uniformity prevents the model from falling into local optima where it only tracks parts of the object nearest to specific keypoints.

2. Geometric-Color Alignment (GCA)

To fuse RGB and Depth data effectively, PAGE establishes a 1-to-1 correspondence by projecting the 3D point cloud onto the 2D image plane. A GC-Fuser (Geometry-guided Color Fuser) then adaptively weights color features based on their geometric relevance. This process filters out redundant or noisy information, such as shadows or lighting variations, that might otherwise degrade geometric precision.

3. Proposal Integration Scoring

Rather than directly regressing coordinates, the network predicts the distance (radius) from each visible point to the keypoints. These predictions serve as "votes" in a 3D accumulator space. The grid cell with the highest score is selected as the keypoint location, providing inherent robustness against occlusions and sensor noise.

Overview of the PAGE pipeline, including keypoint estimation, multi-modal alignment, and proposal scoring.

Experimental Results

PAGE was evaluated on three benchmarks: ArtImage (synthetic), ReArtMix (semi-synthetic), and RobotArm (real-world).

  • Accuracy: On the ArtImage dataset, PAGE achieved a rotation error of just 1.7° for laptops, a significant improvement over the 5.4° error seen in previous state-of-the-art methods like A-NCSH.
  • Real-World Robustness: On a complex 7-part robotic arm, PAGE reduced average rotation error from 12.1° to 7.5°.
  • Geometric Precision: The model demonstrated high 3D Intersection over Union (IoU) scores, particularly for prismatic joints (e.g., drawers), reaching 91.8%.

Conclusion

By shifting from heuristic to pre-defined keypoints and implementing geometry-guided multi-modal alignment, PAGE sets a new standard for articulated pose estimation. While challenges remain regarding error accumulation in deep kinematic chains, the framework's robustness to noise and occlusions marks a significant step forward for robotic manipulation and interaction.

Summary updated

BibTeX
@inproceedings{xu2025pre,
  author = {Wenbo Xu and Li Zhang and Liu Liu and Yan Zhong and Haonan Jiang and Xue Wang and Rujing Wang},
  title = {{Pre-defined Keypoints Promote Category-level Articulation Pose Estimation via Multi-Modal Alignment}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/237},
}
Figure