RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition

Yao Guo, Youfu Li, Zhanpeng Shao

Abstract

The motion behaviors of a rigid body can be characterized by a six degrees of freedom motion trajectory, which contains the 3-D position vectors of a reference point on the rigid body and 3-D rotations of this rigid body over time. This paper devises a rotation and relative velocity (RRV) descriptor by exploring the local translational and rotational invariants of rigid body motion trajectories, which is insensitive to noise, invariant to rigid transformation and scale. The RRV descriptor is then applied to characterize motions of a human body skeleton modeled as articulated interconnections of multiple rigid bodies. To show the descriptive ability of our RRV descriptor, we explore its potentials and applications in different rigid body motion recognition tasks. The experimental results on benchmark datasets demonstrate that our RRV descriptor learning discriminative motion patterns can achieve superior results for various recognition tasks.

Introduction

Recognizing how objects or people move through 3D space is a cornerstone of human-robot interaction and computer vision. However, raw trajectory data is inherently noisy and sensitive to viewpoint changes. If a person performs the same gesture from a different angle, or if sensor data is grainy, standard recognition algorithms often fail.

Existing descriptors generally suffer from two major limitations. First, many ignore rotational data, tracking only the 3D position of a point and missing the crucial "twist" and "turn" information of a rigid body. Second, advanced descriptors often rely on second-order time derivatives (acceleration). In discrete, real-world data, calculating acceleration amplifies noise, leading to poor recognition performance. The RRV (Rotation and Relative Velocity) descriptor addresses these issues by capturing full 6-DOF (degrees of freedom) motion using only first-order derivatives, making it inherently more stable and viewpoint-independent.

Methodology

The core of the proposed system is the RRV descriptor ($S$), which decomposes motion into two invariant components: rotational and translational.

The RRV descriptor pipeline, from raw 6D trajectory data to classification using DTW or SVM.

Rotational and Translational Invariants

To handle viewpoint changes, the authors use Singular Value Decomposition (SVD) to normalize rotation axes. These normalized rotations are represented as reparameterized unit quaternions ($S_r$). Quaternions are preferred over Euler angles because they avoid "Gimbal lock" and provide a more compact representation than rotation matrices.

For translational invariants ($S_t$), the authors adapt the Square-Root Velocity Function (SRVF)—a tool typically used for shape analysis—and project it onto a view-independent local coordinate system. By projecting translational velocity onto the instantaneous rotation matrix, the descriptor captures how rotation physically influences position, providing a richer feature set.

The "Virtual Rigid Body" (VRB) Extension

To apply this rigid-body framework to articulated human skeletons, the authors introduce the concept of Virtual Rigid Bodies. A limb, such as an arm, is modeled as a VRB spanning from a root joint (shoulder) to an end joint (wrist). This abstraction allows the system to capture complex internal dynamics without the need to track every individual joint wiggle, simplifying the recognition of human actions.

Experimental Evaluation

The researchers evaluated the RRV descriptor across three major benchmarks: AUSLAN2 (95 Australian Sign Language signs), MSRAction3D (20 human actions), and MSRC-12 (12 gesture classes).

Visualization of point trajectories (a), rotation vectors (b), and the resulting translational (c) and rotational (d) invariants over time.

As shown in the visualization of invariants, the RRV descriptor successfully extracts smooth, discriminative patterns from raw trajectories. On the MSRAction3D dataset, the RRV-VRB method achieved 93.44% accuracy, outperforming popular methods like Lie Group (89.48%) and Cov3DJ (90.53%). On the AUSLAN2 dataset, it reached 92.56%, a significant improvement over the SoSaLe descriptor (73.27%).

Key Insights and Conclusion

A primary advantage of the RRV descriptor is its noise resilience. By avoiding second-order derivatives, the descriptor maintains high accuracy even when the signal-to-noise ratio is low. While competing methods' performance plummets as noise increases, RRV remains remarkably stable.

Furthermore, the system is highly efficient. While Dynamic Time Warping (DTW) provides high accuracy, a Bag-of-Words (BoW) approach with an SVM classifier can process samples in 13–24ms, making it viable for real-time robotics. Future work may involve integrating these physics-informed invariants into deep learning architectures to further improve training efficiency and robustness against occlusions.

Summary updated

Abstract

Motion behaviors of a rigid body can be characterized by a 6-dimensional motion trajectory, which contains position vectors of a reference point on the rigid body and rotations of this rigid body over time. This paper devises a Rotation and Relative Velocity (RRV) descriptor by exploring the local translational and rotational invariants of motion trajectories of rigid bodies, which is insensitive to noise, invariant to rigid transformation and scaling. A flexible metric is also introduced to measure the distance between two RRV descriptors. The RRV descriptor is then applied to characterize motions of a human body skeleton modeled as articulated interconnections of multiple rigid bodies. To illustrate the descriptive ability of the RRV descriptor, we explore it for different rigid body motion recognition tasks. The experimental results on benchmark datasets demonstrate that this simple RRV descriptor outperforms the previous ones regarding recognition accuracy without increasing computational cost.
BibTeX
@article{guo2016rrv,
  author = {Yao Guo and Youfu Li and Zhanpeng Shao},
  title = {{RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition}},
  journal = {IEEE Transactions on Cybernetics},
  year = {2016},
  doi = {10.1109/tcyb.2017.2705227},
}
Figure