Abstract
Time series classification is a fundamental task in computer vision and signal processing, yet traditional models like Hidden Conditional Random Fields (HCRFs) often struggle with a "parameter bottleneck." As the number of latent states increases to capture complex data structures, the parameter count grows linearly, leading to a high risk of overfitting. This paper presents the Hidden-Unit Logistic Model (HULM), a novel discriminative framework that replaces multinomial latent variables with a collection of binary stochastic hidden units. This approach allows for an exponential increase in latent state representation ($2^H$) with only a linear increase in parameters ($O(H)$), enabling more expressive models that remain computationally efficient and robust against overfitting.
Introduction
Assigning a single label to a sequence of observations—such as identifying a gesture from a video or a word from an audio clip—requires modeling complex temporal dependencies. While HCRFs have been widely used, their reliance on a single $k$-nomial variable at each time step limits their scalability. To increase the model's capacity, one must increase $k$, which directly increases the number of parameters. The Hidden-Unit Logistic Model (HULM) addresses this by using a distributed representation of latent states, decoupling the complexity of the state space from the growth of the parameter set.
The HULM Framework
The core innovation of HULM is its architecture, which utilizes $H$ binary stochastic hidden units at each time step $t$. Unlike traditional models that use a single state variable, HULM employs multiple parallel chains of binary units.
As shown in the figure above, the model consists of input observations $X_1, \dots, X_T$, hidden units $Z_{t,h}$, and a global label $y$. By using $H$ binary units, the model can represent $2^H$ distinct latent states. This distributed coding is significantly more efficient than the local representation used in HCRFs. For instance, representing 1024 states would require a 1024-way multinomial variable in an HCRF, but only 10 binary units in a HULM.
Inference and Learning
To ensure tractability, HULM is designed such that the hidden-unit chains are conditionally independent given the input sequence and the label. This independence allows for exact inference using a standard forward-backward algorithm, which scales linearly with both the sequence length $T$ and the number of hidden units $H$. Furthermore, HULM is a discriminative model, meaning it optimizes the conditional probability $P(y|X)$ directly, focusing its representational power on features that distinguish between classes rather than modeling the entire data distribution.
Experimental Evaluation
The HULM was evaluated on five diverse datasets spanning handwriting, speech, and action recognition, including Online Handwritten Characters (OHC), Arabic Spoken Digits (ASD), and the MSR Action 3D dataset. These datasets range from low-dimensional trajectory data to high-dimensional (570-dimensional) skeletal joint features.
Key Results
HULM consistently outperformed or matched state-of-the-art models like HCRF and Fisher Kernel Learning (FKL). Across all five datasets, it achieved the best average rank (1.8) and the lowest average generalization error (18.75%). Notably, on the Cohn-Kanade Extended (CK+) facial expression dataset, HULM reduced the error rate from 11.04% to 6.44%. It also demonstrated superior performance in detecting subtle facial Action Units, achieving state-of-the-art AUC and F1-scores.
Conclusion
The Hidden-Unit Logistic Model provides a powerful alternative to traditional latent-state models for time series classification. By leveraging binary stochastic units, it breaks the parameter bottleneck, allowing for highly expressive latent representations without the risk of overfitting associated with high-dimensional parameter spaces. Future work may involve extending the model to capture longer-range temporal dependencies through higher-order Markov chains or integrating it into multi-task learning frameworks for simultaneous sequence labeling and classification.
Summary updated