A Novel Skeleton-Based Human Activity Discovery Technique Using Particle Swarm Optimization with Gaussian Mutation

Parham Hadikhani, Daphne Teck Ching Lai, Wee-Hong Ong

TL;DR

This paper introduces HPGMK, a novel unsupervised method for discovering human activities from 3D skeleton data. By combining advanced feature extraction, dimensionality reduction with PCA, and a hybrid PSO-K-means clustering algorithm with Gaussian mutation, HPGMK effectively groups similar activities without requiring labeled data, achieving superior performance on challenging datasets.

What Problem Does This Paper Address?

Human Activity Recognition (HAR) is a crucial field with applications ranging from human-computer interaction to intelligent transportation. However, most existing HAR systems rely on supervised learning, requiring vast amounts of meticulously labeled data. This is a significant bottleneck in real-world scenarios where labeling is time-consuming, expensive, and often impractical due to the sheer volume and variety of human activities. The paper specifically tackles the "human activity discovery" phase, which aims to group similar activities without prior knowledge of their labels. This is akin to how a child learns to differentiate actions without explicit instruction. The challenge lies in extracting meaningful representations from raw data and then clustering these representations effectively, especially when dealing with unsegmented, continuous motion sequences.

Methods

The proposed framework, HPGMK (Hybrid Particle Swarm Optimization with Gaussian Mutation and K-means), operates in two main stages: pre-processing and discovery.

Pre-processing: * Keyframe Selection: Instead of using all frames, important frames are selected based on their "kinetic energy" (derived from joint displacements). This helps capture salient postures and reduce redundancy. * Feature Extraction: A comprehensive set of features is extracted from informative joints (e.g., hands, feet, hips, shoulders, elbows, knees) to capture various aspects of human movement. These include: * Spatial and Temporal Displacement: Measures how joints move within a frame and over time. * Statistical Features: Mean and standard deviation of time-domain features to capture activity evolution. * Orientation and Angle Features: Derived from bone rotations and angles between joints to represent posture and movement. * Dimensionality Reduction: Principal Component Analysis (PCA) is applied to reduce the dimensionality of the extracted features, making the subsequent clustering more efficient and effective. * Sampling: Overlapping sliding windows are used to segment the processed skeleton sequences into activity samples, ensuring that transitions between activities are not missed.

Discovery: * Hybrid PSO with Gaussian Mutation and K-means (HPGMK): This is the core clustering algorithm. * Particle Swarm Optimization (PSO): Used for its global search capability to find potential cluster centers. * Gaussian Mutation: Applied to the global best particle in PSO to enhance exploration and introduce diversity, preventing premature convergence to local optima. * K-means: Integrated to refine the cluster centroids obtained by PSO. This leverages K-means' fast local search capability to overcome PSO's slow convergence near the optimum and improve the exploitation of the algorithm.

Datasets

The proposed HPGMK method was evaluated on five publicly available 3D skeleton-based human activity datasets:

  • Cornell Activity Dataset (CAD-60)
  • UTKinect-Action3D (UTK)
  • Florence3D (F3D)
  • Kinect Activity Recognition Dataset (KARD)
  • MSR DailyActivity3D (MSR)

These datasets vary in the number of activities, subjects, and environments, presenting challenges such as intra-class variation and similar postures.

Key Insights and Ideas

  • Unsupervised Discovery is Crucial: The paper strongly emphasizes the need for unsupervised methods in human activity discovery due to the impracticality of labeling large, diverse datasets in real-world applications.
  • Comprehensive Feature Extraction Matters: Combining spatial, temporal, statistical, and orientation features provides a richer representation of human activities, leading to better discrimination compared to methods that focus on only one aspect.
  • Keyframe Selection Reduces Noise: Identifying and using only the most informative frames based on kinetic energy helps to filter out redundant or noisy data, improving efficiency and accuracy.
  • Hybrid Clustering for Robustness: The combination of PSO's global search with K-means' local refinement, enhanced by Gaussian mutation for diversity, creates a powerful clustering mechanism that avoids common pitfalls like local optima and slow convergence.
  • PCA for Efficiency and Effectiveness: Dimensionality reduction via PCA not only speeds up the clustering process but also improves accuracy by focusing on the most discriminative features.

Key Results

  • Superior Accuracy: HPGMK achieved superior performance across all five datasets compared to state-of-the-art unsupervised methods, including ENSC, SSC, and deep clustering approaches. The average overall accuracy was reported as 77.53% (CAD-60), 56.54% (UTK), 66.84% (F3D), 46.02% (KARD), and 40.12% (MSR).
  • Improved Performance on Challenging Data: HPGMK demonstrated better performance on activities with high intra-class variation, corrupted data, and those performed in both sitting and standing positions.
  • Efficiency Gains: The combination of PCA and keyframe selection significantly reduced clustering time. The paper also shows that combining components of HPGMK (PSO, K-means, mutation, PCA, keyframes) progressively improves convergence speed and clustering time.
  • Statistical Significance: Kruskal-Wallis tests confirmed a statistically significant difference in performance between HPGMK and several other methods (KM, PSO, SSC, ENSC, SDCN) across most datasets.

Limitations and Future Work

  • Pre-configured Number of Clusters: A limitation acknowledged is that the number of clusters (activities) needs to be pre-configured. The authors suggest extending HPGMK to automatically estimate the number of activities.
  • Outlier Detection: The paper notes that outliers can negatively impact cluster centers. Future work could incorporate outlier detection methods to reject such data points and improve cluster formation.
  • Manual Parameter Tuning: The current PSO structure relies on manual parameter tuning based on expertise. Automating this setup process would make the method more adaptable to various datasets and circumstances.
  • Deep Learning Integration: While the paper focuses on a non-deep learning approach as a baseline, future work could explore integrating deep learning techniques for feature extraction or other aspects of the framework.

Summary updated

Abstract

Human activity discovery aims to cluster the activities performed by humans without any prior information on what defines each activity. Most methods presented in human activity recognition are supervised, where there are labeled inputs to train the system. In reality, it is difficult to label activities data because of its huge volume and the variety of human activities. This paper proposes an unsupervised framework to perform human activity discovery in 3D skeleton sequences. First, an approach for data pre-processing is presented. In this stage, important frames are selected based on kinetic energy. Next, the displacement of joints, statistical displacements, angles, and orientation features are extracted to represent the activities information. Since not all extracted features have useful information, the dimension of features is reduced using PCA. Most methods proposed for human activity discovery are not fully unsupervised. They use pre-segmented videos before categorizing activities. To deal with this, we have used a sliding time window to segment the time series of activities with some overlapping. Then, activities are discovered by our proposed Hybrid Particle swarm optimization (PSO) with Gaussian Mutation and K-means (HPGMK) algorithm to provide diverse solutions. PSO is used due to its straightforward idea and powerful global search capability which can identify the ideal solution in a few iterations. Finally, k-means is applied to the outcome centroids from each iteration of the PSO to overcome the slow convergence rate of PSO. The experiment results on five datasets show that the proposed framework has superior performance in discovering activities compared to the other state-of-the-art methods and has increased accuracy of at least 4% on average.
BibTeX
@article{hadikhani2023novel,
  author = {Parham Hadikhani and Daphne Teck Ching Lai and Wee-Hong Ong},
  title = {{A Novel Skeleton-Based Human Activity Discovery Technique Using Particle Swarm Optimization with Gaussian Mutation}},
  journal = {IEEE Trans. Hum. Mach. Syst.},
  year = {2023},
  url = {https://openreview.net/forum?id=tyuBiu6XY3},
}
Figure