Real-Time Detection of Moving Objects from Shaking Camera Based on the Multiple Background Model and Temporal Median Background Model

Taeho Kim, Kang-Hyun JoJournal of Institute of Control Robotics and Systems 2010 · DOI

Abstract

In this paper, we present the detection method of moving objects based on two background models. These background models support to understand multi layered environment belonged in images taken by shaking camera and each model is MBM (Multiple Background Model) and TMBM (Temporal Median Background Model). Because two background models are Pixel-based model, it must have noise by camera movement. Therefore correlation coefficient calculates the similarity between consecutive images and measures camera motion vector which indicates camera movement. For the calculation of correlation coefficient, we choose the selected region and searching area in the current and previous image respectively then we have a displacement vector by the correlation process. Every selected region must have its own displacement vector therefore the global maximum of a histogram of displacement vectors is the camera motion vector between consecutive images. The MBM classifies the intensity distribution of each pixel continuously related by camera motion vector to the multi clusters. However, MBM has weak sensitivity for temporal intensity variation thus we use TMBM to support the weakness of system. In the video-based experiment, we verify the presented algorithm needs around 49(ms) to generate two background models and detect moving objects.

Introduction

Background subtraction is a fundamental technique for detecting movement, but it typically assumes a static camera. When a camera shakes—due to wind, unstable mounting, or being handheld—the entire image shifts. Standard algorithms often interpret this global shift as massive foreground movement, resulting in significant noise. This paper presents a robust approach to this challenge by combining motion estimation with a dual-layer background modeling strategy, achieving real-time performance even with significant camera jitter.

Methodology

The proposed system operates in three main stages to filter out noise caused by camera movement while remaining sensitive to actual moving targets.

Camera Motion Estimation

To compensate for camera shake, the algorithm calculates a displacement vector between consecutive frames. It uses a correlation coefficient between a "selected region" in the current frame and a "searching area" in the previous frame.

Illustration of the motion estimation process using a selected region and searching window to determine displacement vectors.

By creating a histogram of these vectors across the image, the system identifies the dominant global motion (the camera shake). This global maximum of the histogram serves as the camera motion vector.

Dual Background Modeling

  1. Multiple Background Model (MBM): This pixel-based model represents each pixel with multiple clusters of intensity values. These clusters are linked across frames using the estimated motion vector, allowing the model to "track" the background as it drifts across the sensor.
  2. Temporal Median Background Model (TMBM): While MBM provides long-term stability, it can be slow to adapt to sudden changes. The TMBM calculates the median intensity of a pixel over a short window (11 frames), acting as a "short-term memory" to quickly recognize newly revealed background areas.
Temporal intensity variation I(t) at a single pixel, highlighting the need for a model that can handle sudden changes.

The final detection is a logical combination of these two models, ensuring that only objects moving independently of the camera are flagged.

Experimental Results

The algorithm was evaluated using custom-captured video sequences, including handheld "shaking" footage (8,438 frames at 320x240 resolution).

  • Processing Speed: The entire pipeline takes approximately 49ms per frame, enabling real-time operation at roughly 20 frames per second.
  • Accuracy: In handheld tests, the method successfully detected pedestrians and vehicles with significantly less noise than standard MBM implementations. The TMBM effectively eliminated "ghosting" and false positives occurring when a moving camera reveals previously occluded background.

Conclusion and Future Work

The hybrid MBM-TMBM approach provides a robust solution for real-time object detection from unstable cameras. Future work will explore moving from grayscale to the HSI color space to improve sensitivity for objects with intensities similar to the background. Additionally, expanding motion estimation to include camera rotation and 3D movements remains a key area for development.

Summary updated

Abstract

In this paper, we present the detection method of moving objects based on two background models. These background models support to understand multi layered environment belonged in images taken by shaking camera and each model is MBM(Multiple Background Model) and TMBM (Temporal Median Background Model). Because two background models are Pixel-based model, it must have noise by camera movement. Therefore correlation coefficient calculates the similarity between consecutive images and measures camera motion vector which indicates camera movement. For the calculation of correlation coefficient, we choose the selected region and searching area in the current and previous image respectively then we have a displacement vector by the correlation process. Every selected region must have its own displacement vector therefore the global maximum of a histogram of displacement vectors is the camera motion vector between consecutive images. The MBM classifies the intensity distribution of each pixel continuously related by camera motion vector to the multi clusters. However, MBM has weak sensitivity for temporal intensity variation thus we use TMBM to support the weakness of system. In the video-based experiment, we verify the presented algorithm needs around 49(ms) to generate two background models and detect moving objects.
BibTeX
@article{kim2010real,
  author = {Taeho Kim and Kang-Hyun Jo},
  title = {{Real-Time Detection of Moving Objects from Shaking Camera Based on the Multiple Background Model and Temporal Median Background Model}},
  journal = {Journal of Institute of Control Robotics and Systems},
  year = {2010},
  doi = {10.5302/j.icros.2010.16.3.269},
}
Figure