Fire Video Recognition Based on Local and Global Adaptive Enhancement

Jian Ding, Yun Yi, Tinghua Wang, Tao TianAlgorithms 2024 · DOI

TL;DR

This paper introduces LGAE, a novel deep learning architecture for fire video recognition that effectively captures both local and global motion characteristics of flames. To support this research, they also present LFVR, a large-scale dataset for fire video recognition, which is crucial for advancing the field.

What Problem Does This Paper Address?

Fires pose a significant threat, and accurate, timely detection is critical. While computer vision has made strides in fire warning systems, existing methods often overlook the dynamic, complex motion patterns inherent in flames. These patterns, influenced by factors like wind, temperature, and fuel type, are key to distinguishing fires from other visual phenomena. Furthermore, the field of fire video recognition suffers from a lack of large-scale, diverse datasets, hindering the development and evaluation of robust models.

Methods

The core of this work is the Local and Global Adaptive Enhancement (LGAE) network. LGAE builds upon a backbone network (specifically, the Swin-T transformer) and introduces two novel blocks:

  • Local Adaptive Motion Enhancement (LAME) block: This block is designed to capture fine-grained local motion information within flame sequences. It employs a series of 3D convolutional layers with varying kernel sizes and dilation to extract multi-scale local features. A key aspect is its adaptive enhancement mechanism, which refines local features.
  • Global Adaptive Motion Enhancement (GAME) block: This block focuses on understanding the broader motion context of flames. It utilizes an adaptive weight layer to learn global motion knowledge and then employs linear layers to process and enhance this global information. This helps in understanding the overall distribution and movement patterns of the fire.

LGAE integrates these LAME and GAME blocks with the backbone to jointly learn both local and global motion cues, aiming to improve fire recognition accuracy.

Datasets

The authors introduce the Large-scale Fire Video Recognition (LFVR) dataset.

  • Size: 11,560 video clips.
  • Content: Includes 1000 long videos of fires (various sizes and types like kitchen, factory, house, wildfire) and 120 long videos of non-fire scenes containing objects that might be mistaken for fires (e.g., lights, red objects, lanterns).
  • Annotation: Clips are 2 seconds long, without overlap, and categorized as "fire" or "non-fire." A rigorous annotation process involving three annotators was used to ensure quality.
  • Splits: The dataset is split into three training-test sets (approximately 7:3 ratio) to mitigate data bias.

The LFVR dataset is made publicly available. The paper also uses the FireNet dataset for comparative evaluation.

Key Insights and Ideas

  • Motion is Key: The paper strongly emphasizes that both local and global motion characteristics of flames are crucial for accurate fire recognition, a factor often underutilized in prior work.
  • Complementary Enhancement: The LAME and GAME blocks are shown to be complementary. While each individually improves performance over the baseline, their combined use in LGAE yields the best results, suggesting that capturing both fine-grained and broad motion patterns is essential.
  • Dataset Importance: The creation of a large-scale, diverse dataset like LFVR is highlighted as a significant contribution, addressing a critical need in the research community for robust model development and evaluation.
  • Challenging Benchmark: The LFVR dataset is presented as a challenging benchmark, with some examples misclassified by even advanced models, indicating room for further research.

Key Results

  • On the newly introduced LFVR dataset, LGAE achieved an F1 score of 88.93%.
  • On the FireNet dataset, LGAE achieved an F1 score of 93.18%.
  • Ablation studies showed that both LAME and GAME blocks individually improved the performance of the baseline VST network. The combined LGAE model consistently outperformed VST, VST+LAME, and VST+GAME across different data splits.
  • In comparisons with other state-of-the-art methods (I3D, TimeSformer, VST) on LFVR, LGAE achieved the highest F1 score (88.93%).
  • On FireNet, LGAE also achieved the best F1 score (93.18%).
  • While LGAE had slightly higher computational complexity (testing time) compared to VST, the improvement in F1 score (2.52%) was deemed worthwhile.

Limitations and Future Work

  • Challenging Dataset: The paper acknowledges that LFVR is a challenging dataset, with some examples misclassified by both VST and LGAE, indicating that further improvements are needed.
  • Smoke Detection: The authors plan to expand the LFVR dataset to include a "smoke" category and develop algorithms that can simultaneously recognize both smoke and flames, as smoke can obscure flames and pose a challenge for current fire detection methods.
  • Computational Cost: While LGAE offers improved accuracy, its testing time is slightly higher than some baseline methods. Future work might focus on optimizing this aspect without sacrificing performance.

Summary updated

BibTeX
@article{ding2024fire,
  author = {Jian Ding and Yun Yi and Tinghua Wang and Tao Tian},
  title = {{Fire Video Recognition Based on Local and Global Adaptive Enhancement}},
  journal = {Algorithms},
  year = {2024},
  doi = {10.3390/a18010008},
}
Figure