Adaptive Object Detection and Visibility Improvement in Foggy Image

Nan Dong, Zhen Jia, Jie Shao, Жипенг Ли, Fuqiang Liu, Jianwei Zhao, Pei-Yuan PengJournal of Multimedia 2011 · DOI

TL;DR

This paper introduces a novel algorithm for detecting objects in foggy surveillance footage and simultaneously enhancing their visibility. It achieves this by first measuring the fog's intensity and then employing an atmospheric scattering model to isolate and improve only the detected foreground objects, significantly reducing computational cost.

What Problem Does This Paper Address?

Visual surveillance systems are crucial for public safety, but their effectiveness is severely hampered by adverse weather conditions like fog. Fog causes significant image degradation, leading to a loss of contrast and detail, which conventional object detection algorithms, designed for clear images, fail to handle. While existing methods exist for removing fog or enhancing visibility, they are often computationally expensive and require processing the entire image, making real-time application challenging. This paper addresses the need for an efficient, integrated approach to detect and enhance objects in foggy surveillance footage without the prohibitive cost of full-image fog removal.

Methods

The proposed method consists of several key components:

  • Fog Property Measurement: A novel metric is introduced to automatically determine the degree of fogginess in an image. This metric is based on the "Dark Channel Prior" theory, which observes that in non-sky image patches, the minimum intensity across color channels (the dark channel) is typically very low. In foggy images, this dark channel value increases due to atmospheric light. By comparing the dark channel prior of a current frame to that of a clear reference frame, the algorithm quantifies the fog's influence. This allows the system to decide whether to apply the fog-specific processing or proceed with standard methods.
  • Object Detection using Atmospheric Scattering Model: For foggy images, the core object detection relies on an atmospheric scattering model. This model describes how light from a scene point is attenuated and scattered as it travels to the observer. The algorithm first computes a "background depth map" from two reference images taken under different weather conditions. This background map is computed only once. Then, it calculates an "observed depth map" for the current foggy frame. By comparing the observed depth map to the background depth map, foreground objects (which have different depths than the background) can be identified. This method is capable of detecting both newly appearing objects and objects that have remained stationary ("sojourned").
  • Region-Specific Visibility Enhancement: Crucially, instead of processing the entire image, the visibility enhancement is applied only to the detected foreground object regions. This significantly reduces computational load. The enhancement involves de-hazing the object region using its estimated depth value, followed by standard image enhancement techniques like bilateral filtering and histogram equalization to improve contrast and brightness.

Datasets

The paper states that the experiments were conducted using surveillance videos. Specific datasets are not named, but the videos were captured from outdoor surveillance applications, including traffic surveillance systems. The authors mention that these videos were extracted from original videos captured at 25 fps with a frame size of 720x480.

Key Insights and Ideas

  • Fog as a Depth Cue: The paper cleverly leverages the atmospheric scattering model, which inherently relates fog density to depth, to perform object detection. This is a departure from traditional methods that treat fog purely as an obstruction.
  • Selective Processing for Efficiency: The decision to enhance only the detected object regions, rather than the entire frame, is a critical insight for achieving real-time performance in computationally intensive tasks like fog removal.
  • Adaptive Approach: The integration of a fog measurement metric allows the system to adapt its processing pipeline, using standard methods for clear images and the specialized fog algorithm only when necessary.
  • Detecting Sojourned Objects: The depth-map-based detection method proves effective in identifying objects that have stopped moving, a common challenge for traditional background subtraction methods that might incorrectly classify them as background.

Key Results

  • Effective Fog Measurement: The fogginess measurement metric, based on the dark channel prior, was shown to correlate with fog density, with higher dark channel values indicating heavier fog.
  • Successful Object Detection in Fog: The proposed method successfully detected foreground objects in foggy images, which were missed by conventional background subtraction techniques.
  • Improved Visibility: The visibility enhancement applied to detected objects significantly improved their appearance, making them clearer and more discernible.
  • Computational Efficiency: By processing only object regions, the method significantly reduces computational cost compared to full-image fog removal and subsequent detection. The paper emphasizes this efficiency as a key advantage.
  • Detection of Sojourned Objects: The method demonstrated the ability to detect objects that have stopped moving, which is a limitation of many standard background subtraction algorithms.

Limitations and Future Work

  • Detection Speed: The authors acknowledge that improving detection speed for real-time applications is a future research direction.
  • Video Fog Removal: While the paper focuses on object detection and enhancement, the authors suggest that video fog removal itself is a promising area for providing clearer surveillance records under bad weather.
  • Real-time Performance: While the selective processing contributes to efficiency, achieving true real-time performance across all conditions might still be a challenge.
  • Parameter Tuning: The paper mentions the need for setting thresholds (e.g., for fog measurement), which might require careful tuning for optimal performance across diverse scenarios.

Summary updated

Abstract

Detecting objects of interest and obtaining their clear visual appearances are critical requirements for visual surveillance systems. In this paper we propose a novel algorithm to detect foreground objects from video sequences with fog and then enhance their visibilities. First, we propose a novel metric to measure the image fog property to decide whether the image scene is obscured by fog or not. Second, if there is heavy fog in the scene, a novel approach for object detection based on an atmospheric scattering model is proposed. This novel approach can be used to detect not only newly entering objects but also sojourned objects. Once the foreground object is detected, we enhance its visibility only to avoid processing the whole image. Our proposed algorithm is tested with some surveillance video under different fog conditions. Experimental results show that the proposed approach is efficient and efficient for foreground object detection and visibility enhancement under fog weather conditions.
BibTeX
@article{dong2011adaptive,
  author = {Nan Dong and Zhen Jia and Jie Shao and Жипенг Ли and Fuqiang Liu and Jianwei Zhao and Pei-Yuan Peng},
  title = {{Adaptive Object Detection and Visibility Improvement in Foggy Image}},
  journal = {Journal of Multimedia},
  year = {2011},
  doi = {10.4304/jmm.6.1.14-21},
}
Figure