CCTV Object Detection with Background Subtraction and Convolutional Neural Network

Youngmin Kim, Jiyoung Lee, Illo Yoon, Taekjin Han, Chulyeon KimKIISE Transactions on Computing Practices 2018 · DOI

TL;DR

This paper proposes a method to improve real-time object detection in outdoor CCTV footage by combining background subtraction with Convolutional Neural Networks (CNNs). By first identifying potential objects with background subtraction and then classifying them with a CNN, the authors achieve a balance between accuracy and computational efficiency, demonstrating promising results for crime prevention applications.

What Problem Does This Paper Address?

The increasing prevalence of crime in public spaces has led to a surge in the deployment of CCTV systems. However, current surveillance centers often struggle with the sheer volume of data, with operators unable to effectively monitor a large number of cameras. Existing intelligent CCTV analysis techniques, while promising, often suffer from high false alarm rates or excessive computational demands, hindering their practical implementation. This paper specifically targets the challenge of accurately detecting and classifying objects, particularly people, in outdoor CCTV environments, which are prone to varying lighting conditions, occlusions, and complex backgrounds, making traditional methods less effective.

Methods

The proposed method employs a two-stage approach:

  1. Background Subtraction: This initial stage uses pixel differences between consecutive frames to quickly identify moving objects. Techniques like erosion and dilation are applied to remove noise, resulting in candidate object regions. This process is computationally efficient, achieving approximately 30fps on a CPU, making it suitable for real-time processing.
  2. Convolutional Neural Network (CNN) Classification: The candidate objects extracted from the background subtraction stage are then fed into a CNN for classification. The authors designed a CNN architecture with two convolutional layers, two pooling layers, and two fully connected layers, culminating in a softmax output layer. They experimented with different input image sizes (32x32, 60x60, 120x120) and adjusted the number of feature maps and fully connected layer nodes accordingly. The CNN was trained to distinguish between "person" and "non-person" categories.

Datasets

The authors constructed a custom CCTV dataset by filming in locations prone to crime, such as alleys and playgrounds. This dataset includes images captured from various angles and under different conditions. They collected 8373 images containing people (full or partial) and 1117 images of non-people. For training and evaluation, they created two sets:

  • Same Cameras: Data collected from two specific cameras, randomly split into training and testing sets.
  • Different Cameras: Data collected from entirely different cameras to assess the model's generalization capabilities.

Key Insights and Ideas

  • Hybrid Approach for Efficiency: Combining the speed of background subtraction with the classification power of CNNs offers a practical solution for real-time object detection, reducing computational load compared to end-to-end CNN approaches.
  • Handling Partial Objects: The method demonstrates the ability to detect and classify objects even when only parts of the body are visible (e.g., legs, torso), which is crucial for real-world CCTV scenarios where full body shots are not always available.
  • Importance of Data Diversity and Generalization: The significant drop in accuracy when testing on data from different cameras highlights the challenge of domain adaptation in object detection. This underscores the need for diverse training datasets that capture a wide range of scenarios and environments.
  • Impact of Image Size and Padding: The study explored the effect of input image size and padding on performance, finding that larger image sizes generally improved accuracy for the same camera data, while the benefit of padding was less pronounced.

Key Results

  • Within-Camera Accuracy: The model achieved a classification accuracy of approximately 80% when tested on data from the same cameras used for training.
  • Cross-Camera Accuracy: When tested on data from different cameras, the accuracy dropped to approximately 67.5%.
  • Processing Speed: The proposed method achieved a processing speed of around 15fps, significantly faster than Faster R-CNN (approximately 2.7fps) on the same hardware.
  • Handling Difficult Cases: The system successfully classified some challenging cases, such as partial body views and objects that were misidentified as moving due to shadows or reflections.

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Size and Diversity: The current dataset, while custom-built, may not be sufficiently diverse to cover all possible scenarios, leading to potential overfitting and limited generalization. The imbalance between "person" and "non-person" categories was also addressed by undersampling the "person" class.
  • Misclassification of Similar Objects: The model sometimes misclassified objects like strollers as people, likely due to their presence in training data alongside people. Similarly, objects with low color contrast to the background were also challenging.
  • Lack of Object Tracking: The current method does not utilize object tracking information, which could further improve detection accuracy by leveraging the continuity of detected objects across frames.

Future work suggested by the authors includes:

  • Expanding the Dataset: Building a more comprehensive and diverse dataset by collecting data from a wider range of locations and scenarios, minimizing redundancy.
  • Incorporating Object Tracking: Integrating object tracking to enhance detection by maintaining object identity and leveraging temporal information.
  • Exploring More Complex Scenarios: Researching object detection and event recognition for various crime-related scenarios.
  • Utilizing Public Datasets: Leveraging publicly available datasets to broaden the scope of training and improve model robustness.

Summary updated

Abstract

본 연구는 영상 분석에서 최근 좋은 연구 성과를 내고 있는 컨볼루션 신경망 (Convolutional Neural Network: CNN) 기법을 실외 CCTV 영상 분석에 적용하여 객체 유형을 분류하는 방법론은 제안한다. 배경 차분 (background subtraction)을 사용하여 찾고자 하는 객체 후보들을 추출해내고 이를 CNN을 이용해 분류함으로써 계산량을 줄이는 효과를 얻는 방법이다. CNN 학습용 CCTV 영상 수집을 위해 범죄 발생이 주로 일어나는 골목길, 놀이터 등에서 촬영한 CCTV 영상 DB를 구축하였으며 우선적으로 사람인 객체만 검출하는 분류기를 학습하였다. 다양한 학습 데이터 사이즈와 세팅에 맞게 실험하였으며 실험 결과 약 80%의 분류 정확도를 보였으며 새로운 CCTV 영상으로 테스트했을 때 약 67.5%의 성능을 보였다.
BibTeX
@article{kim2018cctv,
  author = {Youngmin Kim and Jiyoung Lee and Illo Yoon and Taekjin Han and Chulyeon Kim},
  title = {{CCTV Object Detection with Background Subtraction and Convolutional Neural Network}},
  journal = {KIISE Transactions on Computing Practices},
  year = {2018},
  doi = {10.5626/ktcp.2018.24.3.151},
}
Figure