Abstract
Standard gesture recognition models often treat meaningful variations—such as the width of a "this big" gesture—as random noise. This paper introduces Parametric Hidden Markov Models (PHMMs), which explicitly incorporate a global parameter into the model's structure. This approach allows a system to simultaneously recognize a gesture and accurately estimate its quantifying features, such as size or direction, by treating systematic variation as a fundamental component of the gesture's definition.
Introduction
When humans gesture, they often convey specific quantities. For instance, in the phrase "I caught a fish this big," the spatial distance between the hands is the primary carrier of information. Traditional recognition techniques, such as standard Hidden Markov Models (HMMs) or Dynamic Time Warping (DTW), typically rely on matching input to a fixed prototype. Consequently, the difference between a "small" and "large" version of the same gesture is often misinterpreted as noise or a poor match.
To address this, researchers have previously relied on ad hoc methods or trained numerous separate models for every possible variation. This paper proposes a unified mathematical framework that incorporates these variations directly into the model's definition.
Methodology: Parametric Hidden Markov Models
The authors extend the standard HMM by modifying the output probabilities of the hidden states to account for a parameter vector $\theta$.
In a standard HMM, each state $j$ has a fixed mean for its Gaussian output. In a PHMM, the mean $\hat{\mu}_j$ is defined as a linear function of $\theta$: $$\hat{\mu}_j(\theta) = W_j \theta + \bar{\mu}_j$$ where $W_j$ and $\bar{\mu}_j$ are learned parameters describing how the state's output changes relative to $\theta$.
Training and Testing
Training utilizes an extension of the Baum-Welch algorithm (Expectation-Maximization) to learn the weights $W_j$ from labeled data. During testing, the system performs simultaneous recognition and parameter estimation by finding the $\theta$ that maximizes the likelihood of the observed sequence. This holistic estimation uses the entire temporal context of the movement to filter out jitter and noise.
Experimental Evaluation
The model was evaluated on two distinct datasets:
- Size Gesture ("The Fish"): 30 sequences of a person indicating size with two hands, tracked in 3D at 20Hz using the Stereo Interactive Virtual Environment (STIVE).
- Pointing Gesture: 50 sequences of a person pointing at a 2D plane, tracked at 30Hz. Here, $\theta$ represented the $(x, y)$ coordinates of the target.
Results and Discussion
The PHMM demonstrated high precision and robustness across both tasks. For the size gesture, the model estimated hand distance with an average error of only 0.16 inches across a range of 7.7 to 36.6 inches.
Key findings include:
- Data Efficiency: The pointing model maintained an error of ~0.82 inches even when trained on only 5 examples, proving strong generalization.
- Noise Robustness: The PHMM significantly outperformed direct measurement techniques when Gaussian noise was added to input features, as it leverages the entire temporal context of the movement.
- Diagnostic Sensitivity: The training process automatically identifies "stroke" phases as being most sensitive to parameter changes, effectively filtering out preparation and retraction noise.
Conclusion
By embracing variation as information rather than noise, PHMMs provide a robust framework for interpreting parametric gestures. Future work may extend this linear model to non-linear dependencies using neural networks and explore subjective parameters such as "style" or emotional intensity in movement.
Summary updated