Probabilistic Background Subtraction in a Video-based Recognition System

Heesung Lee, Sungjun Hong, Euntai KimKsii Trans. Internet Inf. Syst 2011 · DOI

Abstract

In video-based recognition systems, stationary cameras are used to monitor an area of interest. These systems focus on a segmentation of the foreground in the video stream and the recognition of the events occurring in that area. The usual approach to discriminating the foreground from the video sequence is background subtraction. This paper presents a novel background subtraction method based on a probabilistic approach. We represent the posterior probability of the foreground based on the current image and all past images and derive an updated method. Furthermore, we present an efficient fusion method for the color and edge information in order to overcome the difficulties of existing background subtraction methods that use only color information. The suggested method is applied to synthetic data and real video streams, and its robust performance is demonstrated through experimentation.

Introduction

Background subtraction is a fundamental task in computer vision, serving as the first step for applications ranging from traffic monitoring to surveillance. The primary objective is to distinguish moving objects (foreground) from the static scene (background). However, traditional methods often struggle with two major challenges: temporal instability and camouflage. Temporal instability arises when systems make binary decisions based solely on the current frame, making them vulnerable to sensor noise and flickering. Camouflage occurs when the color of a moving object closely matches the background, leading to false negatives and fragmented detections.

Methodology

The proposed framework addresses these issues through a two-phase process: modeling and detection.

Multi-Feature Modeling

To improve robustness, the system utilizes multiple data types. Instead of relying only on RGB values, it calculates distortions in brightness, chromaticity, and edge magnitude. Edges are particularly valuable because they often remain detectable even when an object's color blends into the background.

Geometric representation of color and brightness distortion in RGB space.

As shown in the color space analysis, the system decomposes the difference between the observed pixel $I_i$ and the expected background $E_i$ into brightness distortion ($\delta_{bi}$) and chromaticity distortion ($\delta_{ci}$). This allows for a more nuanced understanding of how a pixel has changed relative to the background model.

The Belief Framework and Fusion

The core innovation lies in the "belief" framework. Rather than a memoryless threshold, the authors employ a recursive Bayesian update. The state of a pixel is represented as a posterior probability—a "belief"—that it belongs to the foreground, conditioned on the entire history of observations.

Overview of the proposed background subtraction system, illustrating the modeling and detection phases.

The detection phase integrates color and edge information using an algebraic S-norm fusion method. This fuzzy logic-inspired approach ensures that if either feature strongly indicates the presence of a foreground object, the system flags it. Furthermore, the background model is dynamic; it evolves to incorporate stationary objects over time, preventing permanent false positives from moved background elements.

Experimental Evaluation

The method was validated across three distinct scenarios:

  1. Synthetic Data: In simulations with heavy Gaussian noise, the probabilistic classifier achieved 99.1% accuracy, significantly outperforming a standard binary classifier (83.2%).
  2. Real Video Sequences: Indoor footage tests demonstrated the system's ability to handle complex shadows and moving subjects effectively.
  3. The "Wallflower" Benchmark: This standard dataset tests seven specific challenges, including waving trees and sudden illumination changes. The proposed method recorded 8,634 total errors, outperforming established benchmarks like Kim et al. (10,116 errors) and Toyama et al. (11,478 errors).

In terms of performance, the system achieved real-time speeds of approximately 29 FPS on 160x120 video using 2011-era hardware, though higher resolutions (320x240) reduced the frame rate to about 6.6 FPS.

Conclusion

By incorporating temporal memory and multi-feature fusion, this probabilistic approach offers a significant improvement over traditional background subtraction techniques. While the mathematical complexity introduces some computational overhead, the gains in accuracy and noise robustness make it a compelling framework for reliable foreground detection. Future work will focus on optimizing processing speeds and refining sensitivity thresholds for diverse environmental conditions.

Summary updated

BibTeX
@article{lee2011probabilistic,
  author = {Heesung Lee and Sungjun Hong and Euntai Kim},
  title = {{Probabilistic Background Subtraction in a Video-based Recognition System}},
  journal = {Ksii Trans. Internet Inf. Syst},
  year = {2011},
  doi = {10.3837/tiis.2011.04.009},
}
Figure