Unsupervised Clustering of Human Pose Using Spectral Embedding

Muhammad Haseeb, Edwin R. Hancock

Abstract

Recognizing human movement typically requires large-scale labeled datasets for supervised learning. This paper introduces an unsupervised alternative that represents 3D skeleton data as a complex Hermitian matrix. By encoding limb lengths in the real part and angular relationships in the imaginary part, the proposed spectral embedding captures the essential geometry of human poses. Experimental results demonstrate that this approach significantly outperforms traditional Laplacian-based methods, maintaining high clustering accuracy even as the number of distinct poses increases.

Introduction

Categorizing human poses is a fundamental task in computer vision with applications ranging from surveillance to human-computer interaction. While supervised methods have seen great success, they are limited by the need for extensive manual labeling. Unsupervised clustering offers a more scalable solution but faces the challenge of creating a mathematical representation—a "fingerprint"—that is invariant to noise yet sensitive to subtle geometric differences between poses.

Traditional spectral clustering often utilizes the Laplacian matrix of a graph. However, the Laplacian typically only accounts for edge weights (distances), ignoring the directional angular information that defines human posture. This paper addresses this gap by leveraging complex-valued Hermitian matrices to incorporate both distance and orientation into the spectral framework.

Proposed Methodology

The proposed pipeline transforms raw 3D joint coordinates into a robust feature space for clustering.

1. Skeleton to Line Graph

Data is initially captured as a set of 3D coordinates for 20 joints. The researchers transform this skeleton into a "line graph" where limbs (the connections between joints) become the nodes, and the joints themselves become the edges. This shift focuses the analysis on the relationship between limbs, which more closely aligns with human anatomical descriptions of pose.

2. The Hermitian Matrix Representation

The core innovation lies in the construction of a complex Hermitian matrix ($H$). For any two adjacent limbs $i$ and $j$:

  • The Real part of the matrix entry $H_{i,j}$ represents the difference in lengths between the limbs.
  • The Imaginary part encodes the angle subtended by the limbs. Using complex numbers allows for an antisymmetric representation ($\theta_{i,j} = -\theta_{j,i}$), which naturally captures the directed nature of angular geometry.

3. Spectral Embedding and Feature Extraction

The eigenvalues and eigenvectors of the Hermitian matrix are calculated. To ensure the resulting feature vector is of fixed length and invariant to data ordering, the authors apply Symmetric Polynomials (via the Newton-Girard formula) to the complex eigenvectors. Finally, Principal Component Analysis (PCA) is used to project these high-dimensional "pattern vectors" into a 3D space for visualization and final clustering.

Experimental Setup

The method was evaluated using a custom dataset captured via a Microsoft Kinect sensor. The dataset included 200 instances of five distinct poses, characterized by various arm positions while standing.

The five distinct human poses used for evaluation, captured as 3D skeletons.

The data consists of 3D $(x, y, z)$ coordinates for 20 joints, recorded at 30 frames per second. This setup provided a controlled environment to test the sensitivity of the Hermitian embedding to variations in similar poses.

Results and Discussion

The performance of the Hermitian approach was compared against a standard Laplacian spectral method using the Rand Index, where a value of 1.0 indicates perfect clustering relative to ground truth.

Comparison of clustering performance (Rand Index) between the proposed Hermitian method and the standard Laplacian method as the number of poses increases.

As shown in the results, the Hermitian method consistently outperformed the Laplacian baseline. Notably, while the Laplacian method's accuracy dropped significantly as the number of poses increased (falling to 0.6393 for 5 poses), the Hermitian method remained robust, achieving a Rand Index of 0.8617. This suggests that the inclusion of angular information in the imaginary component of the matrix is critical for distinguishing between complex, multi-limb configurations.

Conclusion and Future Work

This research demonstrates that complex-valued spectral embeddings provide a powerful tool for unsupervised pose analysis. By treating the skeleton as a geometric structure in the complex plane, the Hermitian matrix captures nuances that real-valued distance matrices miss.

Future work aims to extend this framework to Quaternions, which would allow for a full 4-component representation of 3D rotations and orientations. Additionally, the researchers plan to test the system's scalability with more dynamic movements and larger, more diverse pose libraries.

Summary updated

Abstract

In this paper we use the spectra of a Hermitian matrix and the coefficient of the symmetric polynomials to cluster different human poses taken by an inexpensive 3D camera, the Microsoft ’Kinect’ for XBox 360. We construct a Hermitian matrix from the joints and the angles subtended by each pair of limbs using the three-dimensional ’skeleton’ data delivered by Kinect. To compute the angles between a pair of limbs we construct the line graph from the given skeleton. We construct pattern vectors from the eigenvectors of the Hermitian matrix. The pattern vectors are embedded into a pattern-space using Principal Component Analysis (PCA). We compere the results obtained with the Laplacian spectra pattern vectors. The empirical results show that using the angular information can be efficiently used to clusters different human poses.
BibTeX
@inproceedings{haseeb2012unsupervised,
  author = {Muhammad Haseeb and Edwin R. Hancock},
  title = {{Unsupervised Clustering of Human Pose Using Spectral Embedding}},
  booktitle = {SSPR/SPR},
  year = {2012},
  doi = {10.1007/978-3-642-34166-3\_51},
}
Figure