Sparse Representation for Kinect Based Hand Gesture Recognition System

Zeke Xu, Zhenhao Huang, Zhuoxiong Zhao, Zhiyuan Li, Pengsen Huang2013 · DOI

Abstract

Hand gesture recognition is a critical component of nonverbal human-computer interaction (HCI). However, traditional vision-based systems often struggle with environmental variability and computational overhead. This paper proposes a novel recognition system that integrates Microsoft Kinect depth-sensing with a sparse auto-encoder (SAE) neural network. By leveraging structural depth data and brain-inspired sparse feature learning, the system achieves a gesture recognition rate of 95% while maintaining computational efficiency.

Introduction

Natural human-computer interaction (HCI) relies heavily on vision-based hand gesture recognition. Despite its potential, the field faces two primary hurdles: environmental sensitivity and computational cost. Standard RGB-based systems are easily compromised by fluctuating lighting conditions, varying skin tones, and complex backgrounds. Furthermore, traditional feature extraction methods like Independent Component Analysis (ICA) are often too slow for real-time applications.

This research addresses these challenges by utilizing the Kinect sensor's depth-sensing capabilities to isolate hand data and employing a sparse auto-encoder to extract robust features efficiently.

Methodology

Kinect-Based Segmentation

To overcome the limitations of color-based tracking, the system utilizes the Kinect’s infrared depth sensor and skeleton tracking. The pipeline identifies the "palm center" and "wrist" from the skeleton map to establish a 3D coordinate for the hand. By applying a specific depth threshold around these points, the system can perfectly isolate the hand from the background, regardless of the user's clothing or the room's ambient light.

The multi-stage segmentation process: identifying skeleton points, capturing the depth map, and isolating the hand gesture.

Sparse Auto-encoder (SAE) Architecture

The core of the recognition engine is a sparse auto-encoder. Unlike standard neural networks, an auto-encoder is designed to reconstruct its input at the output layer. By introducing a "sparseness constraint" to the hidden layer, the network is forced to represent data using only a small subset of active neurons. This architecture mimics the efficiency of the human primary visual cortex.

After training the SAE on 10x10 patches, the network learns to identify fundamental features such as edges and orientations. These learned features are then pooled and processed through a Soft-max regression classifier to categorize the gesture.

Learned feature detectors from the sparse auto-encoder, showing distinct edges and orientations similar to the human visual system.

Datasets and Training

The researchers employed a two-tier data strategy:

  1. Natural Image Dataset: 10 high-resolution natural images were used for initial feature detector training, leveraging the universal nature of local image statistics (like edges).
  2. Custom Hand Gesture Dataset: A dedicated set of 1,000 hand gesture maps (500 for training, 500 for testing) was collected using the Kinect, covering five distinct gesture categories.

Key Results

  • High Accuracy: The system achieved a peak recognition rate of 95% for five gesture categories. Control experiments showed that removing the segmentation step caused a 5% drop in accuracy, highlighting the importance of clean input data.
  • Computational Efficiency: The system could be fully trained in approximately 30 minutes on standard hardware (8GB RAM), proving significantly faster and "cleaner" than traditional ICA methods.
  • Feature Optimization: Testing revealed a "sweet spot" for model complexity; 400 features yielded the highest accuracy (95%), while increasing to 500 features led to a performance dip (89%) due to overfitting.
  • Scalability: When expanded to 20 gesture classes, the system maintained a respectable 90% accuracy rate.

Conclusion

By combining specialized depth-sensing hardware with sparse neural representations, this system provides a robust and efficient solution for hand gesture recognition. The use of the Kinect’s skeleton map effectively eliminates background noise, while the sparse auto-encoder provides a brain-inspired method for high-speed feature extraction. Future work will focus on incorporating more complex properties of the human visual system to further enhance resilience in dynamic environments.

Summary updated

Abstract

Hand gesture recognition that has proven a significant factor to directly influence the nonverbal communication between human and computer is becoming a challenging topic in the field of machine vision. This paper aims to propose a novel hand gesture recognition system which applies sparse representation to the Kinect to improve the efficiency of Kinect-based human-computer interaction. Auto-encoder neural network computation is also utilized to achieve better result. The sparse auto-encoder neural network is versatile and robust in complex features learning and computational efficient. Finally, results indicate that our algorithm greatly facilitates the gesture recognition rate up to 95%.
BibTeX
@misc{xu2013sparse,
  author = {Zeke Xu and Zhenhao Huang and Zhuoxiong Zhao and Zhiyuan Li and Pengsen Huang},
  title = {{Sparse Representation for Kinect Based Hand Gesture Recognition System}},
  year = {2013},
  doi = {10.2991/icaicte.2013.125},
}
Figure