TL;DR
This paper proposes a method to improve real-time object detection in outdoor CCTV footage by combining background subtraction with Convolutional Neural Networks (CNNs). By first identifying potential objects with background subtraction and then classifying them with a CNN, the authors achieve a balance between accuracy and computational efficiency, demonstrating promising results for crime prevention applications.
What Problem Does This Paper Address?
The increasing prevalence of crime in public spaces has led to a surge in the deployment of CCTV systems. However, current surveillance centers often struggle with the sheer volume of data, with operators unable to effectively monitor a large number of cameras. Existing intelligent CCTV analysis techniques, while promising, often suffer from high false alarm rates or excessive computational demands, hindering their practical implementation. This paper specifically targets the challenge of accurately detecting and classifying objects, particularly people, in outdoor CCTV environments, which are prone to varying lighting conditions, occlusions, and complex backgrounds, making traditional methods less effective.
Methods
The proposed method employs a two-stage approach:
- Background Subtraction: This initial stage uses pixel differences between consecutive frames to quickly identify moving objects. Techniques like erosion and dilation are applied to remove noise, resulting in candidate object regions. This process is computationally efficient, achieving approximately 30fps on a CPU, making it suitable for real-time processing.
- Convolutional Neural Network (CNN) Classification: The candidate objects extracted from the background subtraction stage are then fed into a CNN for classification. The authors designed a CNN architecture with two convolutional layers, two pooling layers, and two fully connected layers, culminating in a softmax output layer. They experimented with different input image sizes (32x32, 60x60, 120x120) and adjusted the number of feature maps and fully connected layer nodes accordingly. The CNN was trained to distinguish between "person" and "non-person" categories.
Datasets
The authors constructed a custom CCTV dataset by filming in locations prone to crime, such as alleys and playgrounds. This dataset includes images captured from various angles and under different conditions. They collected 8373 images containing people (full or partial) and 1117 images of non-people. For training and evaluation, they created two sets:
- Same Cameras: Data collected from two specific cameras, randomly split into training and testing sets.
- Different Cameras: Data collected from entirely different cameras to assess the model's generalization capabilities.
Key Insights and Ideas
- Hybrid Approach for Efficiency: Combining the speed of background subtraction with the classification power of CNNs offers a practical solution for real-time object detection, reducing computational load compared to end-to-end CNN approaches.
- Handling Partial Objects: The method demonstrates the ability to detect and classify objects even when only parts of the body are visible (e.g., legs, torso), which is crucial for real-world CCTV scenarios where full body shots are not always available.
- Importance of Data Diversity and Generalization: The significant drop in accuracy when testing on data from different cameras highlights the challenge of domain adaptation in object detection. This underscores the need for diverse training datasets that capture a wide range of scenarios and environments.
- Impact of Image Size and Padding: The study explored the effect of input image size and padding on performance, finding that larger image sizes generally improved accuracy for the same camera data, while the benefit of padding was less pronounced.
Key Results
- Within-Camera Accuracy: The model achieved a classification accuracy of approximately 80% when tested on data from the same cameras used for training.
- Cross-Camera Accuracy: When tested on data from different cameras, the accuracy dropped to approximately 67.5%.
- Processing Speed: The proposed method achieved a processing speed of around 15fps, significantly faster than Faster R-CNN (approximately 2.7fps) on the same hardware.
- Handling Difficult Cases: The system successfully classified some challenging cases, such as partial body views and objects that were misidentified as moving due to shadows or reflections.
Limitations and Future Work
The authors acknowledge several limitations:
- Dataset Size and Diversity: The current dataset, while custom-built, may not be sufficiently diverse to cover all possible scenarios, leading to potential overfitting and limited generalization. The imbalance between "person" and "non-person" categories was also addressed by undersampling the "person" class.
- Misclassification of Similar Objects: The model sometimes misclassified objects like strollers as people, likely due to their presence in training data alongside people. Similarly, objects with low color contrast to the background were also challenging.
- Lack of Object Tracking: The current method does not utilize object tracking information, which could further improve detection accuracy by leveraging the continuity of detected objects across frames.
Future work suggested by the authors includes:
- Expanding the Dataset: Building a more comprehensive and diverse dataset by collecting data from a wider range of locations and scenarios, minimizing redundancy.
- Incorporating Object Tracking: Integrating object tracking to enhance detection by maintaining object identity and leveraging temporal information.
- Exploring More Complex Scenarios: Researching object detection and event recognition for various crime-related scenarios.
- Utilizing Public Datasets: Leveraging publicly available datasets to broaden the scope of training and improve model robustness.
Summary updated