Abstract
In this paper, we address the problem of unsupervised video anomaly detection (UVAD). The task aims to detect abnormal events in test video using unlabeled videos as training data. The presence of anomalies in the training data poses a significant challenge in this task, particularly because they form clusters in the feature space. We refer to this property as the "Anomaly Cluster" issue. The condensed nature of these anomalies makes it difficult to distinguish between normal and abnormal data in the training set. Consequently, training conventional anomaly detection techniques using an unlabeled dataset often leads to sub-optimal results. To tackle this difficulty, we propose a new method called Cleansed k-Nearest Neighbor (CKNN), which explicitly filters out the Anomaly Clusters by cleansing the training dataset. Following the k-nearest neighbor algorithm in the feature space provides powerful anomaly detection capability. Although the identified Anomaly Cluster issue presents a significant challenge to applying k-nearest neighbor in UVAD, our proposed cleansing scheme effectively addresses this problem. We evaluate the proposed method on various benchmark datasets and demonstrate that CKNN outperforms the previous state-of-the-art UVAD method by up to 8.5% (from 82.0 to 89.0) in terms of AUROC. Moreover, we emphasize that the performance of the proposed method is comparable to that of the state-of-the-art method trained using anomaly-free data.
Introduction: The Anomaly Cluster Challenge
Unsupervised video anomaly detection (UVAD) is a critical task in surveillance, aiming to identify abnormal events using only unlabeled training data. Unlike traditional one-class classification where training data is guaranteed to be normal, UVAD must contend with anomalies present in the training set.
A significant hurdle identified in this work is the "Anomaly Cluster" issue. In video data, an abnormal event—such as a vehicle driving on a pedestrian path—typically persists across multiple consecutive frames. This temporal redundancy causes anomalies to form dense clusters in the feature space, making them appear statistically "normal" to standard density-based or distance-based detection algorithms.
Proposed Method: Cleansed k-Nearest Neighbor (CKNN)
To address the Anomaly Cluster problem, we propose CKNN, an object-centric framework that cleanses the training data before applying a k-nearest neighbor (k-NN) classifier.
1. Object Cleansing
The cleansing phase identifies and removes likely anomalies from the unlabeled training set. We first extract object candidates using a pre-trained detector (e.g., Mask R-CNN). For each object, we compute "pseudo-anomaly scores" across two streams:
- Appearance Stream: Uses features from an Autoencoder or CLIP to detect visually unusual objects.
- Motion Stream: Uses Optical Flow to identify objects with abnormal movement patterns.
Objects with the highest scores (top $\tau\%$) are discarded, effectively pruning the Anomaly Clusters and leaving a "cleansed" set of normal behaviors.
2. k-NN Inference
The remaining cleansed objects are stored in a feature bank. During testing, an anomaly score for a new object is calculated based on its distance to its $k$ nearest neighbors in the bank. If the distance is high, the object is flagged as abnormal. This simple distance-based approach becomes highly effective once the training data is purified.
Experimental Evaluation
We evaluated CKNN on four benchmarks: UCSD Ped2, CUHK Avenue, ShanghaiTech Campus (STC), and UBnormal. We also employed a rigorous Merge+ protocol to prevent frame leakage between training and testing sets.
Key Results
- State-of-the-Art: CKNN achieved an AUROC of 89.0% on the STC dataset, outperforming the previous unsupervised SOTA (LBR-SPR) by 8.5%.
- Competitive with Supervised Models: Remarkably, CKNN's performance on STC (89.0%) is comparable to state-of-the-art methods trained on manually cleaned, anomaly-free data (90.6%).
- Efficiency: The k-NN search component is highly efficient, capable of running at over 100 FPS, making it suitable for real-time deployment.
Discussion and Conclusion
The success of CKNN highlights that for UVAD, cleansing the training data is often more effective than building complex models designed to be robust to noise. By treating anomalies as "mini-normalities" that form clusters, we can proactively filter them out.
However, the method's reliance on the cleansing threshold $\tau$ and the quality of the initial object detector are known limitations. Future work will focus on automating threshold selection and extending the framework to capture complex multi-object interactions.
Summary updated