A Personalized Hand Gesture Recognition System using Soft Computing Techniques

Moon-Jin Jeon, Jun‐Hyeong Do, Sang Wan Lee, Kwang‐Hyun Park, Zeungnam BienJournal of Korean Institute of Intelligent Systems 2008 · DOI

Abstract

Recently, vision-based hand gesture recognition techniques have been developed for assisting elderly and disabled people to control home appliances. Frequently occurred problems which lower the hand gesture recognition rate are due to the inter-person variation and intra-person variation. The recognition difficulty caused by inter-person variation can be handled by using user dependent model and model selection technique. And the recognition difficulty caused by intra-person variation can be handled by using fuzzy logic. In this paper, we propose multivariate fuzzy decision tree learning and classification method for a hand motion recognition system for multiple users. When a user starts to use the system, the most appropriate recognition model is selected and used for the user.

Introduction

Vision-based gesture recognition is a promising interface for multi-user environments, particularly for assisting those with limited mobility in controlling smart home appliances. However, real-world deployment faces two primary hurdles: inter-person variation, where different users perform the same gesture differently, and intra-person variation, where a single user's movements fluctuate due to fatigue, mood, or posture. Traditional "hard" decision boundaries often fail to capture this inherent human "fuzziness." This paper introduces a personalized system that leverages Soft Computing—specifically Multivariate Fuzzy Decision Trees (MFDT)—to create robust, user-specific recognition models without requiring extensive initial training for every new user.

Proposed Methodology

The system architecture follows a structured pipeline from raw video input to appliance control. The process begins with segmenting face and hand regions from the video stream. Using stereo vision, the system acquires a 3D trajectory of the hand. Velocity information is then utilized to precisely extract the start and end points of a gesture, ensuring that only intentional movements are processed for feature extraction.

System pipeline for personalized hand gesture recognition

Multivariate Fuzzy Decision Trees (MFDT)

At the core of the classification engine is the MFDT. Unlike standard decision trees that split data based on a single attribute (univariate splits), MFDT utilizes a vector of attributes to create multivariate splits. The system employs Linear Discriminant Analysis (LDA) to determine the optimal attribute vector that maximizes information gain at each node.

Comparison between univariate (left) and multivariate (right) decision boundaries

As shown in the figure above, multivariate splits allow for diagonal decision boundaries that better fit the natural distribution of gesture data compared to the "staircase" boundaries of univariate trees. By integrating Fuzzy Logic, the system establishes "soft" boundaries (membership functions) instead of rigid thresholds. This approach allows the model to handle the natural noise and variation in human movement more gracefully than traditional methods.

Model Selection and Garbage Rejection

To achieve personalization without long calibration phases, the system maintains a "Model Pool" of pre-trained MFDTs. Upon detecting a new user, a Maximum Likelihood Model Comparison is performed on a small sample of gestures to select the best-fitting model from the pool.

Furthermore, to prevent accidental triggers in a home environment, the authors implemented a state automata-based "Garbage Motion Rejection" layer. This layer filters out common daily activities—such as drinking water, reading a book, or eating snacks—that might otherwise be misclassified as gestures.

Experimental Evaluation

The researchers validated the MFDT algorithm using UCI benchmark datasets (Iris and Wine) and a custom hand gesture dataset involving 10 users.

Dataset Details: * Gestures: 10 distinct motions including Up, Down, Left, Right, Forward, Backward, Circles, and Semicircles. * Features: 16 features extracted from 3D trajectories, including path length, min/max coordinates, and timing. * Garbage Data: 5 common daily activities recorded to test rejection capabilities.

Key Results

  • Algorithm Robustness: MFDT outperformed standard C4.5 and C5.0 algorithms on UCI benchmarks, achieving 96.5% accuracy on the Iris dataset and 89.9% on the Wine dataset.
  • Gesture Accuracy: The personalized system reached an average recognition accuracy of 93.4% across all users.
  • False Positive Mitigation: The system successfully rejected 83.9% of "garbage" motions, demonstrating high reliability for real-world home automation.

Conclusion

By embracing "fuzziness" as a feature rather than a limitation, this research demonstrates that personalized gesture recognition can be both accurate and efficient. The combination of LDA-based multivariate splits and a pre-trained model pool provides a viable path for deploying assistive technologies in diverse, multi-user household environments. Future work will focus on expanding the diversity of the model pool and adapting these principles to modern high-dimensional sensing technologies.

Summary updated

Abstract

최근 하지가 불편한 노약자나 장애인이 집안의 다양한 가전기기를 손쉽게 제어하기 위한 비전 기반의 손동작 인식 기술이 발전해 왔다. 다수의 사용자가 하나의 손동작 인식 시스템을 사용할 경우 사용자마다 손동작 특성이 모두 다르기 때문에 특정 사용자의 인식률이 저하되는 문제가 발생한다. 또한 동일한 사용자라 하더라도 시간에 따라 손동작 특성이 변화할 수 있다. 사용자마다 다른 손동작 특성은 모델 학습 및 선택 기법을 사용해 효과적으로 다루어질 수 있다. 시간에 따라 변하는 사용자의 특성은 퍼지 개념을 이용해 효과적으로 다루어질 수 있다. 본 논문에서는 다변량 퍼지 의사 결정트리를 이용해 사용자 별 인식모델을 만드는 방법을 제시한다. 또한 새로운 사용자가 시스템을 사용할 경우 가장 적합한 모델을 선택해 인식에 사용하고 인식률을 측정한다. Recently, vision-based hand gesture recognition techniques have been developed for assisting elderly and disabled people to control home appliances. Frequently occurred problems which lower the hand gesture recognition rate are due to the inter-person variation and intra-person variation. The recognition difficulty caused by inter-person variation can be handled by using user dependent model and model selection technique. And the recognition difficulty caused by intra-person variation can be handled by using fuzzy logic. In this paper, we propose multivariate fuzzy decision tree learning and classification method for a hand motion recognition system for multiple users. When a user starts to use the system, the most appropriate recognition model is selected and used for the user.
BibTeX
@article{jeon2008personalized,
  author = {Moon-Jin Jeon and Jun‐Hyeong Do and Sang Wan Lee and Kwang‐Hyun Park and Zeungnam Bien},
  title = {{A Personalized Hand Gesture Recognition System using Soft Computing Techniques}},
  journal = {Journal of Korean Institute of Intelligent Systems},
  year = {2008},
  doi = {10.5391/jkiis.2008.18.1.053},
}
Figure