Detecting Unknown Behaviors by Pre-defined Behaviours: An Bayesian Non-parametric Approach

Jin Watanabe, Takatomi Kubo, Fan Yang, Kazushi Ikeda

Abstract

Automatic mouse behavior recognition systems are essential for reducing manual workloads in biological research. However, existing supervised and semi-supervised methods typically operate under a closed-set assumption, failing to identify novel or "undefined" behaviors. This paper introduces the Semi-supervised Infinite Gaussian Mixture Model (SsIGMM), a Bayesian non-parametric framework designed to incorporate labeled information while remaining open to discovering new behavior clusters. By leveraging a modified Chinese Restaurant Process with "cannot-link" constraints, SsIGMM effectively segments and labels mouse-behavior videos, outperforming traditional models in both synthetic and real-world scenarios.

Introduction

In behavioral biology, researchers often analyze mouse videos to study the effects of pharmacological or genetic interventions. Automated systems for this task generally fall into three categories:

  1. Supervised models: Highly accurate for predefined behaviors but prone to misclassifying novel actions into known categories.
  2. Unsupervised models: Capable of discovering new patterns but requiring intensive post-hoc manual labeling.
  3. Standard semi-supervised models: Often constrained by a fixed number of categories, preventing the discovery of undefined behaviors.

The authors address this "open-set" challenge by proposing a model that uses existing labels to guide learning while retaining the flexibility to spawn new clusters for previously unseen behaviors.

Methodology: The SsIGMM Framework

The core of the proposed approach is the SsIGMM, an extension of the Infinite Gaussian Mixture Model (IGMM). Unlike standard mixture models that require a fixed number of components, SsIGMM utilizes a Dirichlet Process (DP) prior, allowing the number of clusters to grow dynamically with the data.

Comparison of the graphical models for the standard IGMM (a) and the proposed SsIGMM (b).

The primary innovation in SsIGMM is the modification of the Chinese Restaurant Process (CRP) to incorporate label information. The model enforces a "cannot-link" constraint: if two data points have different labels (e.g., "Walk" vs. "Groom"), they are forbidden from occupying the same cluster. Conversely, unlabeled data points are free to either join an existing "known" cluster or initiate a brand-new "undefined" cluster if they are statistically distinct from the predefined categories.

Inference is performed using a Collapsed Gibbs Sampler, which iteratively assigns data points to clusters based on the statistical distribution of the features and the constraints imposed by the labels.

Experimental Setup

The model was validated using two distinct datasets:

  1. Synthetic Data: 1,500 points from 10 two-dimensional Gaussian components to test the model's ability to handle multimodal distributions where a single behavior class might consist of multiple clusters.
  2. Real Mouse Behavior: Video data from top and side views. Features included motion of the center of gravity, nose movement, body length, and height. The study predefined three behaviors (Walk, Rear, Groom) while leaving three others undefined (Freeze, Stretch Attend Posture, and Escape) to test discovery capabilities.
Illustration of the assignment process where labeled data (blue/green) and unlabeled data (grey) are assigned to clusters based on available assignments and label constraints.

Key Results and Insights

The performance was evaluated using the Adjusted Rand Index (ARI), where a score of 1.0 indicates perfect alignment with ground truth.

  • Synthetic Performance: SsIGMM achieved an ARI of 0.96, significantly higher than the unsupervised IGMM (0.61) and standard semi-supervised approaches (0.62).
  • Real-World Accuracy: On the mouse behavior dataset, the model reached an ARI of 0.92.
  • Discovery of the Unknown: Qualitative analysis through confusion matrices showed that while baseline models often merged "Escape" or "Freeze" into known categories like "Walk," SsIGMM successfully isolated these into distinct, undefined clusters.

One of the most significant advantages of SsIGMM is its statistical efficiency. By allowing one label to be represented by multiple Gaussian clusters, the model captures the natural variability of biological actions—recognizing that "walking" may look different depending on speed or posture—without losing the ability to flag truly novel behaviors.

Conclusion and Future Work

SsIGMM provides a robust framework for behavior recognition that balances the guidance of labeled data with the discovery potential of unsupervised learning. Future work may involve integrating deep learning-based feature extraction (such as DeepLabCut) to replace manual feature engineering and exploring variational inference to improve computational speed for larger datasets. Despite a note in the original abstract regarding a potential withdrawal of the submission, the technical framework remains a significant contribution to Bayesian non-parametric modeling in ethology.

Summary updated

Abstract

An automatic mouse behavior recognition system can considerably reduce the workload of experimenters and facilitate the analysis process. Typically, supervised approaches, unsupervised approaches and semi-supervised approaches are applied for behavior recognition purpose under a setting which has all of predefined behaviors. In the real situation, however, as mouses can show various types of behaviors, besides the predefined behaviors that we want to analyze, there are many undefined behaviors existing. Both supervised approaches and conventional semi-supervised approaches cannot identify these undefined behaviors. Though unsupervised approaches can detect these undefined behaviors, a post-hoc labeling is needed. In this paper, we propose a semi-supervised infinite Gaussian mixture model (SsIGMM), to incorporate both labeled and unlabelled information in learning process while considering undefined behaviors. It also generates the distribution of the predefined and undefined behaviors by mixture Gaussians, which can be used for further analysis. In our experiments, we confirmed the superiority of SsIGMM for segmenting and labelling mouse-behavior videos.
BibTeX
@misc{watanabe2019detecting,
  author = {Jin Watanabe and Takatomi Kubo and Fan Yang and Kazushi Ikeda},
  title = {{Detecting Unknown Behaviors by Pre-defined Behaviours: An Bayesian Non-parametric Approach}},
  year = {2019},
  eprint = {1911.10806},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.10806},
}
Figure