A Novel Background Modeling Based on Keyframe and Particle Shape Property for Surveillance Video

Yong Fan, Xiu He, Yiyi Lin, Zhanchuan CaiIEEE Access 2023 · DOI

Introduction

Background subtraction is a fundamental task in computer vision, essential for separating moving foreground objects from static backgrounds in surveillance feeds. However, traditional algorithms frequently struggle with intermittent motion—scenarios where objects move slowly or remain stationary for extended periods (e.g., a person sitting on a bench). These situations often lead to "ghosting," where stationary objects are incorrectly absorbed into the background model or leave artifacts behind once they move.

This paper introduces BM-KP (Background Modeling based on Keyframe and Particle shape properties), a novel algorithm designed to address these challenges by treating frame differences as "particles" with specific geometric properties and utilizing a strategic keyframe selection process.

The BM-KP Methodology

The proposed BM-KP model operates through a three-stage pipeline designed for both robustness and computational efficiency.

The BM-KP algorithm pipeline, illustrating keyframe extraction, particle characterization, and background updating stages.

1. Keyframe Extraction and Grouping

Processing every frame in a high-resolution video is often redundant and computationally expensive. BM-KP identifies "representative" keyframes by calculating the standard deviation of differences between adjacent frames. This selective approach reduces noise and minimizes the data redundancy that typically contributes to ghosting artifacts.

2. Particle Shape Characterization

The core innovation of BM-KP lies in how it evaluates motion. The authors treat the binarized difference between frames as a collection of "particles." Instead of relying solely on pixel intensity, they use a 4-tuple of geometric properties to describe these particles:

  • Total size of selected particles.
  • Maximum ratio of Euler number to area, which measures topology and the presence of holes.
  • Smallest ratio of area to circumscribed rectangle.
  • Smallest roundness.

By applying the Pauta Criterion (a statistical outlier detection method) to these properties, the model can effectively filter out "bad" data—such as sensor noise or compression artifacts—that would otherwise corrupt the background estimate.

3. Strategic Background Updating

Rather than updating the background pixel-by-pixel in isolation, BM-KP employs a multi-site comparison strategy. It assesses similarity across five specific locations: the mask boundary, the adjacent exterior, the adjacent interior, the object center, and the interior mask area. This approach recognizes that the boundary of a suspected object is often more informative than its center; if the area just outside a mask matches the input frame, it provides a strong signal that the background is visible and the mask should be updated.

Experimental Results

The authors evaluated BM-KP on the SBMnet (Scene Background Modeling Network) and SBI (Scene Background Initialization) benchmarks.

  • Intermittent Motion Performance: BM-KP achieved state-of-the-art results, ranking 2nd out of 31 methods in the SBMnet "intermittent motion" category.
  • Robustness: Qualitative analysis demonstrated the model's ability to reconstruct clean backgrounds in challenging environments, such as offices with long-term stationary occupants or busy subway stations.
  • Efficiency: The method achieved processing speeds of 5.38 FPS on SBMnet and 10.94 FPS on SBI. While not yet full real-time for high-frame-rate video, it significantly outperforms many high-accuracy models that operate at less than 1 FPS.
  • Accuracy: On the SBI dataset, BM-KP outperformed the BI-GAN algorithm in 26 out of 30 metrics across six test sequences.

Conclusion and Future Work

BM-KP offers a robust solution for background modeling in complex surveillance scenarios, particularly those involving intermittent motion. By leveraging geometric "particle" properties and intelligent keyframe selection, it achieves a high balance between accuracy and speed. Future research will focus on incorporating unsupervised learning techniques to automatically tune the model's parameters, further enhancing its adaptability to diverse environmental conditions like swaying foliage or extreme clutter.

Summary updated

BibTeX
@article{fan2023novel,
  author = {Yong Fan and Xiu He and Yiyi Lin and Zhanchuan Cai},
  title = {{A Novel Background Modeling Based on Keyframe and Particle Shape Property for Surveillance Video}},
  journal = {IEEE Access},
  year = {2023},
  doi = {10.1109/access.2023.3329816},
}
Figure