FlameFinder: Illuminating Obscured Fire through Smoke with Attentive Deep Metric Learning

Hossein Rajoli, Sahand Khoshdel, Fatemeh Afghah, Xiaolong Ma

Abstract

FlameFinder is a deep metric learning (DML) framework designed to accurately detect flames obscured by smoke using thermal imagery from firefighter drones. While traditional RGB cameras are blinded by smoke, thermal cameras can capture obscured flame features but often suffer from false positives due to a lack of absolute thermal reference points. FlameFinder addresses this by utilizing paired thermal-RGB images for training to learn latent flame features from smoke-free samples. By incorporating an attention mechanism to balance multiple DML losses—Center Loss (CL), Triplet Center Loss (TCL), and Triplet Cosine Center Loss (TCCL)—the model achieves superior class discrimination and interpretability. Experimental results on the FLAME2 and FLAME3 datasets demonstrate that FlameFinder outperforms standard backbones by up to 7% in accuracy and significantly reduces false negatives in obscured scenarios.

Introduction

Early wildfire detection is paramount for disaster mitigation. Unmanned Aerial Vehicles (UAVs) equipped with sensors offer a flexible monitoring solution, yet smoke presents a significant hurdle. RGB cameras provide high resolution but cannot penetrate smoke. Thermal cameras can "see" through smoke but rely on relative thermal gradients, leading to false positives from non-fire "hot spots" like sun-heated rocks or warm smoke plumes. The challenge lies in creating a model that recognizes the structural "signature" of a flame—its shape, distribution, and texture—rather than just looking for the brightest pixels.

The FlameFinder Architecture

The proposed framework shifts from simple classification to Deep Metric Learning (DML). Instead of a binary fire/no-fire output, DML maps images into an embedding space where similar features cluster together and dissimilar ones are pushed apart.

The FlameFinder framework overview, illustrating the training pipeline with multi-modal data and the inference pipeline for obscured thermal images.

The system employs a "Multi-Modal Training, Uni-Modal Inference" strategy. During training, an automated algorithm called FlameSeg uses clear RGB images to generate precise labels for paired thermal frames. This "indirect supervision" allows the model to learn what a flame looks like in the thermal domain without requiring massive manually-labeled thermal datasets. During deployment, the model operates solely on thermal data to identify flames through smoke.

Attentive Deep Metric Learning

A core innovation of FlameFinder is its Attention Module. Standard DML often suffers from "loss domination," where one objective, such as Center Loss (CL), overwhelms others. CL minimizes the distance between samples and their class center, but it can miss features sensitive to angular or relative distances.

The attention mechanism dynamically weights three distinct losses:

  • Center Loss (CL): Ensures intra-class compactness.
  • Triplet Center Loss (TCL): Maximizes inter-class separation by ensuring samples are closer to their own class center than to others.
  • Triplet Cosine Center Loss (TCCL): Focuses on the angle between feature vectors, enhancing discrimination in high-dimensional spaces.

By preventing any single loss from dominating, the attention mechanism acts as a regularizer, forcing the model to learn more robust, non-magnitude features such as geometric and textural relationships.

Experimental Results

FlameFinder was evaluated using the FLAME2 (53,000 frames) and FLAME3 datasets. The model consistently outperformed established backbones like ResNet18 and VGG19.

Visual demonstration of FlameFinder's inference on obscured thermal data, showing successful flame detection through smoke.

Key performance metrics include:

  • Accuracy: Achieved 97.8% on FLAME2 and 88.9% on FLAME3, representing a 4.4% to 7% improvement over baseline models.
  • Reliability: The False Negative Rate was reduced to 11.6% (compared to 17% for ResNet18), which is critical for early detection where missed fires can be catastrophic.
  • Clarity: t-SNE visualizations confirmed that FlameFinder creates tighter, more distinct clusters for flames versus non-flame hot spots compared to standard classification approaches.

Conclusion

FlameFinder demonstrates that combining thermal imaging with attentive deep metric learning can effectively overcome the limitations of traditional wildfire detection. By learning the structural signature of flames through indirect RGB supervision and a balanced loss framework, the system provides a reliable tool for UAV-based monitoring in smoky environments. Future work will explore the integration of temporal data to analyze the "flicker" of flames, further distinguishing them from static thermal anomalies.

Summary updated

Abstract

FlameFinder is a deep metric learning (DML) framework designed to accurately detect flames, even when obscured by smoke, using thermal images from firefighter drones during wildfire monitoring. Traditional RGB cameras struggle in such conditions, but thermal cameras can capture smoke-obscured flame features. However, they lack absolute thermal reference points, leading to false positives.To address this issue, FlameFinder utilizes paired thermal-RGB images for training. By learning latent flame features from smoke-free samples, the model becomes less biased towards relative thermal gradients. In testing, it identifies flames in smoky patches by analyzing their equivalent thermal-domain distribution. This method improves performance using both supervised and distance-based clustering metrics.The framework incorporates a flame segmentation method and a DML-aided detection framework. This includes utilizing center loss (CL), triplet center loss (TCL), and triplet cosine center loss (TCCL) to identify optimal cluster representatives for classification. However, the dominance of center loss over the other losses leads to the model missing features sensitive to them. To address this limitation, an attention mechanism is proposed. This mechanism allows for non-uniform feature contribution, amplifying the critical role of cosine and triplet loss in the DML framework. Additionally, it improves interpretability, class discrimination, and decreases intra-class variance. As a result, the proposed model surpasses the baseline by 4.4% in the FLAME2 dataset and 7% in the FLAME3 dataset for unobscured flame detection accuracy. Moreover, it demonstrates enhanced class separation in obscured scenarios compared to VGG19, ResNet18, and three backbone models tailored for flame detection.
BibTeX
@article{rajoli2024flamefinder,
  author = {Hossein Rajoli and Sahand Khoshdel and Fatemeh Afghah and Xiaolong Ma},
  title = {{FlameFinder: Illuminating Obscured Fire through Smoke with Attentive Deep Metric Learning}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2024},
  doi = {10.1109/tgrs.2024.3440880},
}
Figure