Two-Step Real-Time Night-Time Fire Detection in an Urban Environment Using Static ELASTIC-YOLOv3 and Temporal Fire-Tube

Minji Park, ByoungChul Ko

TL;DR

This paper introduces a novel two-step approach for real-time night-time fire detection in urban environments. By combining a modified YOLOv3 (ELASTIC-YOLOv3) for initial fire candidate detection with a temporal "fire-tube" analysis using optical flow and a Bag-of-Features (BoF) representation, the method achieves high accuracy and reduced processing time compared to existing deep learning methods, particularly for challenging low-light conditions.

What Problem Does This Paper Address?

Fires in urban areas, especially at night, pose a significant threat due to increased casualties and property damage. While camera-based fire detection systems offer advantages over traditional sensors, existing methods struggle with the unique challenges of night-time detection. These challenges include the loss of color information, the presence of ambient lighting (headlights, neon signs, streetlights) that can mimic fire characteristics, and the difficulty in distinguishing subtle fire movements from background noise. Current deep learning approaches, while powerful, often require substantial computational resources, making real-time application difficult, and can be prone to false positives in complex night-time scenarios. This paper aims to bridge this gap by developing a robust and efficient system specifically tailored for real-time night-time urban fire detection.

Methods

The proposed method employs a two-step approach:

  1. Fire Candidate Detection (ELASTIC-YOLOv3): The first step utilizes a modified YOLOv3 architecture, termed ELASTIC-YOLOv3. This modification incorporates an "ELASTIC block" into the YOLOv3's base network (Darknet-53). The ELASTIC block is designed to improve the detection of objects of various sizes, particularly small fires, by employing a dual-path structure with upsampling and average pooling to preserve feature information. This enhances the detection rate for small fires without significantly increasing computational complexity.
  2. Temporal Fire-Tube Analysis and Verification (HoF + BoF + Random Forest):
    • Fire-Tube Generation: Candidate fire regions identified by ELASTIC-YOLOv3 are tracked across consecutive frames to form "fire-tubes." This process uses a skip-frame mechanism to efficiently capture temporal information, focusing on frames where significant changes occur within the potential fire region.
  3. Histogram of Optical Flow (HoF): The motion within each fire-tube is analyzed using optical flow. A spatial pyramid pooling (SPP) technique is applied to extract a 45-dimensional HoF feature vector from each fire-tube, capturing the orientation and magnitude of flame movements.
  4. Bag-of-Features (BoF): To represent the temporal characteristics efficiently, the HoF features are converted into a BoF histogram. This involves K-means clustering to create a visual codebook, and then mapping each fire-tube's HoF to the closest visual word in the codebook.
  5. Random Forest (RF) Classifier: Finally, a Random Forest classifier is trained on these BoF histograms to distinguish between actual fire and fire-like regions. RF is chosen for its fast processing speed and ability to handle tabular data effectively, making it suitable for real-time verification.

Datasets

The authors constructed a new dataset called NightFire-DB for training and evaluating their method. This dataset comprises night-time fire and fire-like videos captured in urban environments, including roads, factories, shopping malls, parks, and gas stations. It includes:

  • Training Data: 14 fire and 10 non-fire videos from the NightFire-DB, plus new night-time fires downloaded from YouTube. For ELASTIC-YOLOv3 training, 4000 static images of night-time fires and fire-like objects were used. For temporal analysis, 600 fire-tubes were randomly selected (300 fire, 300 non-fire).
  • Test Data: 10 fire and 10 non-fire videos from the NightFire-DB.

The dataset's average video length was 1030 frames at 30 Hz, with resolutions ranging from 320x240 to 640x480. The authors note that existing datasets like VisFire and KMU Fire & Smoke Database were unsuitable due to their focus on daytime and nearby fires.

Key Insights and Ideas

  • Night-time fire detection is fundamentally different: The paper highlights that the visual cues for fire at night are significantly different from daytime fires, making direct application of daytime algorithms ineffective. The challenges of low light and confounding visual elements are central.
  • Combining spatial and temporal information is crucial: The two-step approach effectively leverages both the spatial detection capabilities of CNNs (ELASTIC-YOLOv3) and the temporal dynamics of fire movement (fire-tube, HoF, BoF) for robust verification.
  • ELASTIC block enhances small object detection: The proposed ELASTIC block within YOLOv3 is shown to improve the detection of small fires, which are often the earliest indicators of a fire.
  • Fire-tubes capture dynamic motion: The temporal fire-tube generation, combined with HoF, provides a powerful way to represent the irregular and dynamic nature of flames, which is critical for distinguishing them from static or moving non-fire objects.
  • Random Forest offers efficient verification: Using a Random Forest classifier on the BoF features provides a computationally efficient way to verify fire candidates, enabling real-time performance.

Key Results

  • Improved Recall: The proposed ELASTIC-YOLOv3 achieved a significant improvement in recall compared to the original YOLOv3, with a 10.2% increase for small fires and 6.1% for medium-sized fires. This indicates fewer missed fires (false negatives).
  • High F1-Scores: The method achieved high F1-scores across all fire sizes, demonstrating a good balance between precision and recall. For instance, the average F1-score for ELASTIC-YOLOv3 was 97.9%, compared to 94.2% for YOLOv3.
  • Competitive Processing Time: While slightly slower than the original YOLOv3 (16 ms vs. 15.9 ms per frame), the proposed method was significantly faster than other state-of-the-art deep learning approaches like Faster R-CNN and SSD, while offering superior recall.
  • Superior Temporal Verification: Compared to ELASTIC-YOLOv3 combined with 3D ConvNets or LSTM for temporal verification, the proposed fire-tube + BoF + RF method achieved a higher F1-score (98.0% vs. 89.3% and 86.6%) and recall (99.5% vs. 89.3% and 99.5%), with a faster processing time (24.8 ms vs. 20.4 ms and 56 ms, noting the 15ms of ELASTIC-YOLOv3 is included in the 24.8ms).
  • Effective Night-time Detection: The method demonstrated successful detection of real fires even with surrounding lights and detected small fires at a distance.

Limitations and Future Work

  • GPU Dependency: The current ELASTIC-YOLOv3 component requires a GPU, limiting its application on low-end embedded systems. The authors suggest that an optimized "tiny" version of YOLOv3 running on a CPU could address this.
  • Dataset Scope: The current experiments were conducted on videos collected from YouTube. The authors acknowledge that performance might vary in real-world urban environments due to changes in weather, camera conditions, and diverse surroundings.
  • Future Research:
    • Developing new types of fire-tube and BoF algorithms.
  • Extending the method to detect forest fires at night using long-distance cameras.
  • Optimizing the algorithm for embedded systems by exploring CPU-based versions.
  • Collecting more diverse urban environment datasets for further analysis.

Summary updated

BibTeX
@inproceedings{park2020two,
  author = {Minji Park and ByoungChul Ko},
  title = {{Two-Step Real-Time Night-Time Fire Detection in an Urban Environment Using Static ELASTIC-YOLOv3 and Temporal Fire-Tube}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=GoTl8fxeOc},
}
Figure