Light-Weight Student LSTM for Real-Time Wildfire Smoke Detection

Mira Jeong, Minji Park, Jae-Yeal Nam, ByoungChul Ko

TL;DR

This paper introduces a novel "teacher-student" framework to create a lightweight, real-time wildfire smoke detection system. By training a smaller "student" LSTM model to mimic a larger "teacher" LSTM, they achieve comparable detection accuracy with significantly reduced computational cost, making it more practical for deployment.

What Problem Does This Paper Address?

Wildfires pose a significant threat, and early detection is crucial to minimize damage. While camera-based smoke detection is cost-effective, distinguishing actual wildfire smoke from similar visual phenomena like clouds and fog remains a challenge. Existing deep learning models, particularly LSTMs, are effective but computationally intensive, hindering real-time application. This paper aims to bridge this gap by developing a highly accurate yet computationally efficient wildfire smoke detection system.

Methods

The proposed method involves a two-stage process:

  1. Candidate Smoke Detection:
    • Keyframe Extraction: To handle the slow motion of distant smoke, the system identifies "keyframes" that exhibit significant motion changes, rather than processing every frame.
  2. YOLOv3 Detector: A You-Only-Look-Once (YOLOv3) object detector is used to identify candidate smoke regions within these keyframes. This approach is chosen over simple motion detection to avoid misclassifying clouds, fog, or camera shake.
  3. Wildfire Smoke Verification (Teacher-Student LSTM):
    • Smoke-Tube Construction: A "smoke-tube" is created by collecting a sequence of frames (30 frames over 3 seconds at 10 fps) around the detected candidate region. This captures the temporal and spatial characteristics of the smoke.
  4. Spatial Feature Extraction: A fine-tuned ResNet50 model is used to extract feature vectors from each frame within the smoke-tube.
  5. Teacher-Student Framework: A deep, three-layer LSTM (the "teacher") is trained on a dataset with "soft labels" (probability distributions of classes). This teacher model is then used to generate soft labels for a larger dataset. A shallow, single-layer LSTM (the "student") is trained on these soft labels, aiming to replicate the teacher's performance with fewer parameters. This teacher-student approach allows for model weight reduction while preserving accuracy.

Datasets

  • Candidate Smoke Detector Training: A dataset of 2022 smoke images (excluding non-smoke) was collected.
  • LSTM Verifier Training and Testing: A custom dataset was created, combining wildfire smoke videos and non-wildfire smoke videos (including clouds, fog, and chimney smoke).
  • Video Configuration: Videos had resolutions of 1920x1080 or higher, with a frame rate of 25 fps.
  • Training Data: 70 wildfire videos (12 for testing) and 64 non-wildfire videos (12 for testing) were used. Data was selected to represent various sizes, locations, shapes, and seasons.
  • New Benchmark: The authors note the scarcity of high-quality open benchmark datasets and constructed their own to address this.

Key Insights and Ideas

  • Keyframes for Efficiency: Focusing on keyframes with significant motion is a smart way to reduce computational load for detecting slow-moving, distant smoke.
  • YOLOv3 for Robust Candidates: Using YOLOv3 for candidate detection, rather than just motion, helps to filter out false positives from clouds and fog more effectively.
  • Teacher-Student for Lightweight Models: The teacher-student learning paradigm is a powerful technique for compressing large, accurate models into smaller, faster ones without a significant performance drop. This is crucial for real-time applications.
  • Soft Labels for Better Generalization: Training the student model with soft labels from the teacher model allows it to learn more nuanced class probabilities, leading to better generalization and potentially improved performance on challenging cases.

Key Results

  • Object Detection Performance: YOLOv3 was selected as the object detector, achieving a good balance between recall and processing speed, despite a slightly lower F1-score compared to ELASTIC-YOLOv3.
  • Teacher LSTM Performance: The deep, three-layer LSTM teacher model achieved an F1-score of 87.85%.
  • Student LSTM Performance: The lightweight, single-layer student LSTM achieved an F1-score of 87.39%, demonstrating a minimal performance drop (0.53%) compared to the teacher.
  • Model Compression: The student LSTM model had 8.4 times fewer parameters and a significantly faster processing time per frame (reduced by 0.019s) compared to the teacher LSTM.
  • Accuracy vs. Cells: Experiments showed that increasing the number of cells in the student LSTM beyond 24 did not improve performance and could even hinder learning.

Limitations and Future Work

  • Distinguishing Similar Smoke: The current system still struggles to accurately distinguish between chimney smoke/fog and actual wildfire smoke in some cases.
  • Improving Pre-processing: The authors plan to improve the smoke-tube generation method and LSTM structure to better handle these challenging cases.
  • Reducing False Negatives: Future work will focus on developing a smoke detection model for the pre-processing stage that minimizes false negatives (missing actual smoke).
  • Further LSTM Simplification: Research will continue on further simplifying the LSTM structure for even greater suitability in real-time systems.

Summary updated

BibTeX
@inproceedings{jeong2020light,
  author = {Mira Jeong and Minji Park and Jae-Yeal Nam and ByoungChul Ko},
  title = {{Light-Weight Student LSTM for Real-Time Wildfire Smoke Detection}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=7Qy0_T6uWW},
}
Figure