Abstract
Understanding the dynamic behavior of articulated objects is a prerequisite for autonomous robotic interaction in human environments. We present P^3-Net, a novel framework for part mobility parsing from 3D point cloud sequences. Unlike previous methods that rely on implicit temporal relationships, P^3-Net explicitly learns point correspondences across frames. This approach enables the extraction of robust trajectory features, leading to state-of-the-art performance in motion flow prediction, part segmentation, and kinematic attribute estimation.
Introduction
For a robot to perform tasks such as opening a drawer or a refrigerator, it must understand the mechanical constraints of the object's parts. This "part mobility parsing" involves identifying moving segments and estimating their motion parameters, such as rotation axes for revolute joints or translation vectors for prismatic joints.
Analyzing these properties from point cloud sequences is inherently difficult because point clouds are unordered sets. A point at a specific index in one frame does not necessarily correspond to the same physical location in the next. P^3-Net addresses this by establishing explicit point-to-point matches, allowing the system to track the "history" of individual points as they move through space.
Methodology
The core of P^3-Net is the Match-LSTM, which integrates temporal information by re-ordering features from previous time steps to match the current point distribution. This alignment is driven by a matching matrix computed through a combination of intra-frame and inter-frame attention mechanisms.
To ensure the matching process is both accurate and computationally feasible, the network employs a Gumbel-Sinkhorn module. This module produces a doubly stochastic matrix, enforcing a near-one-to-one correspondence between points in consecutive frames. By stacking these correspondences, the network generates "trajectory features" that characterize the motion path of every point. Finally, a multi-task head predicts three critical outputs:
- Motion Flow: The per-point displacement vector.
- Part Segmentation: The assignment of points to distinct moving or static parts.
- Kinematic Attributes: The mathematical description of the joint (e.g., axis orientation and range of motion).
Experiments and Results
P^3-Net was rigorously tested on the Motion Dataset and PartNet benchmarks. The model demonstrated significant improvements over previous state-of-the-art architectures:
- Motion Flow: Achieved an End-Point Error (EPE) of 0.181, outperforming FlowNet3D.
- Segmentation: Reached a 0.84 IoU, surpassing MeteorNet and ASAP-Net.
- Efficiency: Despite the added complexity of explicit matching, the model trained faster than several implicit competitors due to better-defined feature learning.
Beyond synthetic benchmarks, the authors validated P^3-Net on real-world scans. Even with sensor noise and occlusions typical of real environments, the model successfully identified moving parts and their axes, enabling a simulated robotic arm to interact correctly with drawers and doors.
Conclusion
P^3-Net demonstrates that explicit correspondence learning is a powerful paradigm for 3D temporal analysis. By transforming unordered point sets into coherent trajectories, the network provides the detailed kinematic understanding necessary for complex robotic manipulation. Future work will focus on scaling the matching mechanism to higher-resolution data and handling multiple simultaneously moving parts.
Summary updated