Robust Dual-Graph Regularized Moving Object Detection

Jing Qin, Ruilong Shen, Ruihan Zhu, Biyun Xie

TL;DR

This paper introduces a novel method for robustly detecting moving objects in videos by separating them from the background. It achieves this by modeling the background as a low-rank matrix and the foreground as a sparse matrix, enhanced with dual graph regularizations to capture complex spatial and temporal structures, and solved efficiently using the ADMM framework.

What Problem Does This Paper Address?

Detecting moving objects in videos is crucial for applications like human-robot interaction, enabling robots to safely navigate and collaborate with people. A common challenge is accurately separating moving objects (foreground) from static scenes (background), especially when dealing with varying lighting, shadows, and complex object movements. Traditional methods often struggle with these complexities, leading to inaccurate background models or fragmented foreground detections. This paper aims to improve the robustness and accuracy of this separation process.

Methods

The core of the proposed method lies in a robust dual-graph regularized moving object detection model. The authors decompose a video into a low-rank background component ($L$) and a sparse foreground component ($S$).

  • Weighted Nuclear Norm Regularization: Instead of the standard nuclear norm for low-rankness, they employ a weighted nuclear norm. This allows for more adaptive sparsity in the singular values of the background matrix, promoting better low-rank approximations.
  • Dual Graph Regularization: To capture the intricate spatial and temporal structures of the background, they introduce two graph Laplacians:
  • Spatial Graph Laplacian ($\Phi_s$): Captures spatial relationships between pixels by considering patchwise similarity.
  • Temporal Graph Laplacian ($\Phi_t$): Captures temporal relationships between frames by considering similarity across time. These graph regularizations are applied to the background component ($L$) to enforce spatiotemporal smoothness.

  • $\ell_1$-norm for Foreground: The foreground component ($S$) is regularized using the $\ell_1$-norm, which is well-suited for promoting sparsity.

  • Alternating Direction Method of Multipliers (ADMM): The optimization problem, which involves multiple regularization terms, is solved efficiently using the ADMM framework. This involves introducing auxiliary variables and iteratively updating the components of the model.

Datasets

The paper evaluates its method on two simulated moving object datasets:

  1. Whole Body Movement Video: A video capturing a student walking naturally.
  2. Arm Movement Video: A video of a student rotating their forearm and hand.

These datasets appear to be custom-generated or collected for the purpose of this study, focusing on human motion. The authors mention using RGB video camera data from a Microsoft Azure Kinect Sensor for the experiments.

Key Insights and Ideas

  • Adaptive Low-Rankness is Key: Using a weighted nuclear norm for the background is more effective than a standard nuclear norm, allowing the model to adaptively promote low-rankness based on the data.
  • Geometry Matters: Incorporating both spatial and temporal graph structures significantly improves the background model by capturing the inherent geometric properties of the scene over time.
  • ADMM for Complex Problems: The ADMM framework is a powerful tool for efficiently solving complex optimization problems with multiple, diverse regularization terms, making the proposed model computationally feasible.
  • Robustness to Noise and Shadows: The combination of $\ell_1$-norm for foreground and weighted nuclear norm for background, along with graph regularization, leads to improved robustness against noise and shadows, which are common challenges in real-world videos.

Key Results

The proposed method, "Alg. 1," consistently outperformed other state-of-the-art methods (LAGO, SPCP, SPGL1) in both quantitative and qualitative evaluations on the tested datasets.

  • Background Recovery: Achieved superior background recovery, as evidenced by lower Relative Error (RE) and higher Peak Signal-to-Noise Ratio (PSNR) in Table I and Table II. For the walking video, Alg. 1 achieved an RE of 0.0145 and PSNR of 41.95.
  • Foreground Detection: Demonstrated better foreground detection accuracy, with higher Precision (Pr), Recall (Re), and F-measure (Fm). For the walking video, Alg. 1 achieved Pr=0.9688, Re=0.7187, and Fm=0.8252.
  • Qualitative Results: Visual comparisons in Figure 1 and Figure 2 (walking video) and Figure 3 and Figure 4 (arm motion video) show that the proposed method produces cleaner background reconstructions and more accurate foreground masks with sharper edges compared to the baselines.
  • Running Time: The method maintains a good balance between running time and detection accuracy, with a comparable running time to LAGO (around 5 seconds) while offering superior performance.

Limitations and Future Work

  • Limitations:
    • The paper focuses on gray-scale videos, although it mentions that the approach can be extended to RGB data channelwise.
  • The method might struggle with highly dynamic backgrounds or scenes where the distinction between foreground and background is inherently ambiguous.
  • Future Work:
    • Develop faster methods based on low-rank tensor decompositions.
  • Extend the approach to better separate shadows from detected moving objects under more sophisticated lighting conditions.
  • Explore more advanced graph construction techniques.

Summary updated

Abstract

Moving object detection and its associated background-foreground separation have been widely used in a lot of applications, including computer vision, transportation and surveillance. Due to the presence of the static background, a video can be naturally decomposed into a low-rank background and a sparse foreground. Many regularization techniques, such as matrix nuclear norm, can therefore be imposed on the background. In the meanwhile, sparsity or smoothness based regularizations, such as total variation and $\ell_{1}$, can be imposed on the foreground. Moreover, graph Laplacians are further used to capture the complicated geometry of background images. Recently, weighted regularization techniques including the weighted nuclear norm regularization have been proposed in the image processing community to promote adaptive sparsity while achieving efficient performance. In this paper, we propose a robust dual-graph regularized moving object detection model based on a new weighted nuclear norm regularization and spatiotemporal graph Laplacians, which is solved by the alternating direction method of multipliers (ADMM). Numerical experiments on realistic body movement data sets have demonstrated the effectiveness of this method in separating moving objects from background, and the great potential in robotic applications.
BibTeX
@inproceedings{qin2022robust,
  author = {Jing Qin and Ruilong Shen and Ruihan Zhu and Biyun Xie},
  title = {{Robust Dual-Graph Regularized Moving Object Detection}},
  booktitle = {2022},
  year = {2022},
  doi = {10.1109/icma54519.2022.9856248},
}

Loading related content...

Figure