Abstract
Unsupervised human action categorization is a significant challenge in computer vision due to background clutter and high intra-class variability. This paper introduces the Consensus Information Bottleneck (CIB) method, which bridges the gap between multi-view clustering and consensus clustering. By simultaneously leveraging raw feature data and auxiliary clusterings, CIB learns a robust representation that maximizes mutual information. Experimental results on five benchmark datasets, including HMDB and UCF50, demonstrate that CIB consistently outperforms state-of-the-art unsupervised methods.
Introduction
Categorizing human actions in videos without manual labels is a massive challenge due to factors such as background clutter, camera motion, and high intra-class variation. For instance, two people performing the same action, like "running," can look very different depending on the environment and the actor's physique. Researchers typically use various descriptors like Histograms of Oriented Gradients (HOG), Histograms of Optical Flow (HOF), and Motion Boundary Histograms (MBH) to capture these actions. However, standard strategies for combining these features—multi-view clustering and consensus clustering—often fall short. Multi-view clustering suffers from the "curse of dimensionality," while consensus clustering loses access to the underlying raw feature data, limiting the final result to the quality of initial, potentially biased, clusters.
The Consensus Information Bottleneck (CIB) Method
The authors propose the Consensus Information Bottleneck (CIB) to address these limitations. This approach is rooted in the Information Bottleneck (IB) framework, which seeks to find a compressed representation of data that retains as much "relevant" information as possible.
CIB treats one feature set as the primary "feature variable" and treats other existing clusterings (derived from different features) as "auxiliary variables." The algorithm maximizes the Mutual Information (MI) between the learned categories and both the primary feature and the auxiliary clusterings. This ensures the model searches for a partition consistent with both the raw data and the consensus of other models. Optimization is performed via a sequential draw-and-merge algorithm that iteratively moves data points between clusters to minimize information loss.
Experimental Evaluation
The researchers evaluated CIB on five benchmark datasets representing a range of difficulty:
- Weizmann: 10 actions, 90 videos (controlled environment).
- KTH: 6 actions, 599 sequences (outdoor/indoor).
- UCF Sports: 1100 videos from broadcast television.
- UCF50: 50 categories, 6000 realistic YouTube videos.
- HMDB: A massive, "in-the-wild" dataset with 51 action categories and 6849 clips.
Key Results and Insights
CIB consistently outperformed state-of-the-art methods across all datasets. On the Weizmann dataset, CIB achieved 76.4% accuracy, significantly higher than traditional K-means (52.8%) or standard Information Bottleneck (69.8%). Even on complex "in-the-wild" datasets like HMDB, CIB showed a steady improvement over baseline consensus methods (e.g., 23.3% vs. 19.2% for MCLA).
By focusing on one feature variable at a time while using others as auxiliary clusterings, CIB gains the benefits of multi-view data without the computational explosion of concatenating high-dimensional vectors. Furthermore, the model proved robust to trade-off parameters ($\lambda$), making it easier to deploy in real-world scenarios without exhaustive tuning.
Conclusion
The Consensus Information Bottleneck method provides a robust framework for unsupervised action categorization by bridging the gap between raw feature analysis and consensus clustering. Future work will explore "soft" clustering to capture the nuances of human action and the integration of more heterogeneous features, such as combining visual data with audio or metadata.
Summary updated