Abstract
Automatic mouse behavior recognition systems are essential for reducing manual workloads in biological research. However, existing supervised and semi-supervised methods typically operate under a closed-set assumption, failing to identify novel or "undefined" behaviors. This paper introduces the Semi-supervised Infinite Gaussian Mixture Model (SsIGMM), a Bayesian non-parametric framework designed to incorporate labeled information while remaining open to discovering new behavior clusters. By leveraging a modified Chinese Restaurant Process with "cannot-link" constraints, SsIGMM effectively segments and labels mouse-behavior videos, outperforming traditional models in both synthetic and real-world scenarios.
Introduction
In behavioral biology, researchers often analyze mouse videos to study the effects of pharmacological or genetic interventions. Automated systems for this task generally fall into three categories:
- Supervised models: Highly accurate for predefined behaviors but prone to misclassifying novel actions into known categories.
- Unsupervised models: Capable of discovering new patterns but requiring intensive post-hoc manual labeling.
- Standard semi-supervised models: Often constrained by a fixed number of categories, preventing the discovery of undefined behaviors.
The authors address this "open-set" challenge by proposing a model that uses existing labels to guide learning while retaining the flexibility to spawn new clusters for previously unseen behaviors.
Methodology: The SsIGMM Framework
The core of the proposed approach is the SsIGMM, an extension of the Infinite Gaussian Mixture Model (IGMM). Unlike standard mixture models that require a fixed number of components, SsIGMM utilizes a Dirichlet Process (DP) prior, allowing the number of clusters to grow dynamically with the data.
The "Cannot-Link" Constraint
The primary innovation in SsIGMM is the modification of the Chinese Restaurant Process (CRP) to incorporate label information. The model enforces a "cannot-link" constraint: if two data points have different labels (e.g., "Walk" vs. "Groom"), they are forbidden from occupying the same cluster. Conversely, unlabeled data points are free to either join an existing "known" cluster or initiate a brand-new "undefined" cluster if they are statistically distinct from the predefined categories.
Inference is performed using a Collapsed Gibbs Sampler, which iteratively assigns data points to clusters based on the statistical distribution of the features and the constraints imposed by the labels.
Experimental Setup
The model was validated using two distinct datasets:
- Synthetic Data: 1,500 points from 10 two-dimensional Gaussian components to test the model's ability to handle multimodal distributions where a single behavior class might consist of multiple clusters.
- Real Mouse Behavior: Video data from top and side views. Features included motion of the center of gravity, nose movement, body length, and height. The study predefined three behaviors (Walk, Rear, Groom) while leaving three others undefined (Freeze, Stretch Attend Posture, and Escape) to test discovery capabilities.
Key Results and Insights
The performance was evaluated using the Adjusted Rand Index (ARI), where a score of 1.0 indicates perfect alignment with ground truth.
- Synthetic Performance: SsIGMM achieved an ARI of 0.96, significantly higher than the unsupervised IGMM (0.61) and standard semi-supervised approaches (0.62).
- Real-World Accuracy: On the mouse behavior dataset, the model reached an ARI of 0.92.
- Discovery of the Unknown: Qualitative analysis through confusion matrices showed that while baseline models often merged "Escape" or "Freeze" into known categories like "Walk," SsIGMM successfully isolated these into distinct, undefined clusters.
One of the most significant advantages of SsIGMM is its statistical efficiency. By allowing one label to be represented by multiple Gaussian clusters, the model captures the natural variability of biological actions—recognizing that "walking" may look different depending on speed or posture—without losing the ability to flag truly novel behaviors.
Conclusion and Future Work
SsIGMM provides a robust framework for behavior recognition that balances the guidance of labeled data with the discovery potential of unsupervised learning. Future work may involve integrating deep learning-based feature extraction (such as DeepLabCut) to replace manual feature engineering and exploring variational inference to improve computational speed for larger datasets. Despite a note in the original abstract regarding a potential withdrawal of the submission, the technical framework remains a significant contribution to Bayesian non-parametric modeling in ethology.
Summary updated