TL;DR
This paper introduces a novel framework for video anomaly detection that goes beyond simply flagging suspicious events. It achieves interpretability by analyzing object interactions and individual human poses, allowing it to not only detect anomalies but also explain their root causes. This approach also demonstrates strong cross-domain adaptability, enabling models to generalize to new surveillance scenes without extensive retraining.
What Problem Does This Paper Address?
Traditional video anomaly detection systems often rely on complex, end-to-end trained deep learning models that extract spatiotemporal features. While these methods can be effective at detecting anomalies, they suffer from a significant drawback: a lack of interpretability. This "black-box" nature makes it impossible to understand why an event is flagged as anomalous, hindering the ability to assess its severity or take appropriate action. Furthermore, these models are often difficult to adapt to new surveillance environments, requiring extensive retraining or new annotated data for each scene, which is often impractical. This paper addresses these limitations by proposing a method that provides interpretable anomaly detection and facilitates cross-domain generalization.
Methods
The proposed framework employs a dual-monitoring approach:
- Global Object Monitoring: This branch focuses on understanding interactions between objects in a scene. It constructs "scene graphs" where nodes represent objects and edges represent their relationships (e.g., "person riding bike"). An interaction detection network, trained on the VRD dataset, identifies these relationships. The novelty lies in monitoring the semantic distance of these detected interactions to a set of learned "nominal" interactions. This distance, quantified as $GM_{stat}(t)$, serves as a global anomaly statistic.
- Local Object Monitoring: This branch monitors individual human poses. It uses a pre-trained pose estimator (AlphaPose) to extract skeletal trajectories. These trajectories are then fed into a message-passing encoder-decoder network (inspired by GRUs) to reconstruct human poses. The reconstruction error, $LM_{stat}(t)$, serves as a local anomaly statistic, indicating deviations from normal human actions.
- Sequential Anomaly Detection: The global and local statistics ($GM_{stat}(t)$ and $LM_{stat}(t)$) are combined into a single vector $z_t$. This vector is then fed into a sequential change detection algorithm. The algorithm uses a CUSUM-like approach to monitor the evidence of anomaly over time, providing a decision statistic $s_t$. An anomaly is declared when $s_t$ exceeds a threshold.
- Anomaly Interpretation: The framework can identify which branch (global or local) contributed most to the detected anomaly, thus explaining the root cause (e.g., an unusual object interaction or an abnormal human pose).
A key aspect of the method is the use of semantic embeddings for interactions and a contrastive loss for metric learning, which helps in learning discriminative representations for nominal and anomalous interactions.
Datasets
- Training:
- VRD dataset: Used to train the interaction detection network.
- ShanghaiTech dataset: Used to learn nominal interactions and poses for the global and local monitoring branches, respectively.
- Evaluation:
- ShanghaiTech dataset: Used for evaluating performance.
- CUHK Avenue dataset: Used for evaluating performance.
The paper notes that the UCF-Crime dataset was not used as it's intended for a different formulation of the problem.
Key Insights and Ideas
- Interactions are key to interpretability: The paper argues that understanding how objects interact is crucial for explaining anomalies, rather than just looking at individual object appearances or motions.
- Dual-monitoring for comprehensive detection: Combining global (object interactions) and local (human poses) monitoring provides a more robust and nuanced approach to anomaly detection.
- Semantic embeddings enable generalization: Using semantic embeddings for interactions allows the model to capture relationships that are semantically similar, facilitating cross-domain adaptability.
- Zero-shot cross-domain adaptability is achievable: The proposed method demonstrates the ability to generalize to new surveillance scenes without any specific training data from those scenes, a significant advantage over many existing methods.
- Interpretable anomaly causes: The framework can pinpoint whether an anomaly is due to an unusual interaction between objects or an abnormal human action, providing valuable insights.
Key Results
- Interpretability: The paper demonstrates qualitative interpretability, showing that the model can correctly identify the cause of anomalies (e.g., "person carry bag," "person on bike"). Quantitative interpretability is measured using a Recall@5 metric, achieving 0.373 on the ShanghaiTech dataset.
- Cross-Domain Adaptability: The proposed approach significantly outperforms state-of-the-art methods in terms of frame-level AUC for cross-domain adaptability on both ShanghaiTech and Avenue datasets, achieving an AUC of 0.79 on Avenue and 0.689 on ShanghaiTech.
- Anomalous Frame Localization: The method achieves competitive performance in anomaly localization, with AUCs of 85.78% on CUHK Avenue and 71.18% on ShanghaiTech.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the interpretability performance is measured using Recall@k, and a direct comparison with other methods is not possible as they are not interpretable.
- The cross-domain adaptability relies on the assumption that anomalous behaviors are similar across datasets, which is met in the evaluated datasets.
- Future Work:
- The paper suggests that the proposed framework can be extended to more complex scenarios and different types of anomalies.
- Further exploration of cross-domain adaptability in more diverse and challenging scenarios could be a future direction.
Summary updated