Probabilistic Models for Concurrent Chatting Activity Recognition

Chia-chun Lian, Jane Yung-jen HsuIJCAI 2009

Abstract

Recognizing simultaneous social interactions in crowded environments presents a significant computational challenge. This paper introduces a framework based on Factorial Conditional Random Fields (FCRFs) to model concurrent chatting activities. By incorporating co-temporal relationships and utilizing the Iterative Classification Algorithm (ICA) for inference, the proposed model achieves higher accuracy and significantly reduced computational overhead compared to traditional parallel models and Loopy Belief Propagation (LBP).

Introduction

In social settings like conferences or parties, multiple conversations occur simultaneously. For automated systems to map these social networks, they must identify who is interacting with whom in real-time. The primary difficulty lies in the non-independent nature of these activities; one person's state (e.g., speaking) is highly correlated with their neighbors' states. Traditional Hidden Markov Models (HMMs) struggle with these complex dependencies, either ignoring them through parallel processing or suffering from computational explosion.

Methodology: Factorial Conditional Random Fields

The authors propose using Factorial Conditional Random Fields (FCRFs) to represent the group's state. Unlike standard CRFs that process a single sequence, FCRFs manage multiple layers of labels—one for each participant—simultaneously.

The FCRF model architecture, illustrating Local, Temporal, and Co-temporal cliques that capture dependencies across participants and time.

The model's probability calculations rely on three potential functions:

  1. Local Potentials: Connect individual audio features (e.g., volume) to activity states.
  2. Temporal Potentials: Model state transitions for a single person over time.
  3. Co-temporal Potentials: Capture the relationships between different participants within the same time slice.

To handle the complexity of inference in these graphs, the authors replace the standard Loopy Belief Propagation (LBP) with the Iterative Classification Algorithm (ICA). ICA simplifies the FCRF into linear chains and iteratively updates labels based on neighbor estimates, avoiding the convergence issues common in loopy graphs.

Experimental Evaluation

The framework was tested on two custom auditory datasets collected via wearable recorders:

  • Meeting Activity: 80 minutes of seated discussion among three participants, primarily using audio volume features.
  • Public Occasion: 45 minutes of free movement in a noisy laboratory, utilizing complex features like Mutual Information (MI) and human voice detection.

Results and Discussion

The FCRF model consistently outperformed parallel models (PCRFs). In the "Public Occasion" scenario, FCRF with ICA achieved an F-score of 68.94%, a substantial improvement over PCRFs (50.47%) and HMM-based approaches (~36%).

A critical finding was the efficiency of the ICA inference method. While LBP is the standard for such graphs, ICA proved to be both more accurate and significantly faster, as it avoids the convergence failures that often plague LBP in dense, loopy graphs.

Comparison of computational efficiency between LBP and ICA, showing dramatic reductions in learning and decoding times.

As shown in the experimental results, the learning phase for FCRF with LBP required over 113,000 seconds, whereas ICA completed the same task in just 6,437 seconds—a nearly 18-fold speedup. Decoding for real-time recognition was also twice as fast with ICA (19 seconds vs 40 seconds).

Conclusion

This research demonstrates that modeling social groups as interconnected systems via FCRFs provides vital context for activity recognition. The adoption of ICA addresses the computational bottlenecks of discriminative models, making them viable for real-time applications. Future work will explore scaling these models to larger social groups where the complexity of fully connected nodes may require further optimization.

Summary updated

BibTeX
@inproceedings{lian2009probabilistic,
  author = {Chia-chun Lian and Jane Yung-jen Hsu},
  title = {{Probabilistic Models for Concurrent Chatting Activity Recognition}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2009},
}
Figure