TL;DR
This paper reveals that ~12% of fire detections from the geostationary GOES ABI satellite are spatially misaligned with detections from the higher-resolution VIIRS satellite. Ignoring these misalignments leads to biased accuracy assessments of GOES fire products, significantly overestimating false alarms. The authors propose a buffer-window approach that effectively mitigates these misalignments, providing a more reliable evaluation method.
What Problem Does This Paper Address?
Accurate and timely fire detection is critical for mitigating wildfire damage. Geostationary (GEO) satellites like GOES ABI offer near real-time fire monitoring, while Low Earth Orbit (LEO) satellites like VIIRS provide higher spatial resolution and are often used as a "ground truth" for evaluating GEO fire detection algorithms. However, a key challenge arises from spatial misalignments between detections from these different satellite types. These misalignments, caused by factors like parallax and sensor characteristics, can lead to incorrect assessments of GEO fire detection performance, particularly inflating false alarm rates. This study aims to quantify the prevalence of these misalignments and demonstrate a robust method for evaluating GEO fire products that accounts for them.
Methods
The core of the methodology involves:
- Data Harmonization: Aligning GOES ABI and VIIRS fire detections temporally (within a ±5 minute window) and spatially by rasterizing VIIRS data onto the GOES ABI grid.
- Non-Maximal Suppression: A technique applied to the rasterized VIIRS data to handle sub-pixel misalignments and aggregate detections from neighboring pixels, creating a more robust "ground truth" representation.
- Buffer Window Evaluation: Instead of direct pixel-to-pixel comparison, the authors introduce a spatial buffer window around each GOES FDC detection. A GOES detection is considered a true positive if the buffer contains a VIIRS detection. If the buffer contains no VIIRS detections, it's flagged as a false positive.
- Misalignment Quantification: Categorizing GOES detections into three groups: VIIRS detections inside the GOES pixel, VIIRS-GOES misalignments (detected by one but not the other within a buffer), and GOES false alarms.
- Accuracy Metrics: Calculating precision, recall, and F1 scores using the buffer-based approach, and comparing these with results obtained without buffering.
Datasets
The study utilizes a large dataset of fire detections from:
- GOES-16/17 ABI Fire Detection and Characterization (FDC) product: This is the GEO-based data source.
- VIIRS (Visible Infrared Imaging Radiometer Suite) active fire product (Collection 2, Level 2): This is the LEO-based reference, considered higher resolution and more accurate.
The dataset covers three distinct regions: California-Oregon (high latitude north), Amazonas (equatorial latitude), and Patagonia (high latitude south), encompassing diverse ecosystems and wildfire conditions. The constructed dataset contains approximately 157,000 VIIRS fire detections and 15,000 GOES FDC fire pixels.
Key Insights and Ideas
- Misalignment is Systemic: The ~12% misalignment rate is not an anomaly but a consistent issue across different latitudes and ecosystems, highlighting a fundamental challenge in GEO-LEO fire detection comparison.
- Buffering is Crucial: The simple act of applying a spatial buffer window around GEO detections dramatically improves the reliability of accuracy assessments, particularly by reducing the apparent false alarm rate.
- Pixel-to-Pixel is Misleading: Direct pixel-to-pixel comparisons, common in prior work, are inherently flawed due to geometric and optical distortions, leading to significantly overestimated false alarms.
- Generalizable Methodology: The proposed buffer-window and non-maximal suppression approach is adaptable to other GEO/LEO sensor combinations, offering a standardized way to evaluate fire detection algorithms.
Key Results
- Prevalence of Misalignment: Approximately 12% of GOES FDC fire detections exhibit spatial misalignments with VIIRS detections across all study regions.
- False Alarm Reduction: Without a buffer, the estimated false alarm rate for GOES FDC ranges from 21% to 36%. With a 3x3 buffer window, this rate is reduced to 7-15%.
- Improved Accuracy Metrics: The use of a buffer window significantly increases precision, recall, and F1 scores, indicating a more accurate representation of the GOES FDC's true performance. For example, precision increases from ~0.74-0.79 without a buffer to ~0.85-0.93 with a buffer.
- Detection Probability: The probability of GOES FDC detecting a fire increases with the introduction of a buffer window, demonstrating its effectiveness in capturing detections that would otherwise be missed due to misalignment.
Limitations and Future Work
- Buffer Size Trade-off: While larger buffer windows (e.g., 7x7) further improve some metrics, they increase the risk of matching unrelated fires, especially in high-latitude regions with larger pixel footprints. A smaller window (e.g., 3x3) is recommended to balance accuracy and avoid over-association.
- Small Fire Detection: The GOES FDC product struggles to detect small fires, as indicated by lower recall values when the VIIRS detection threshold is low (t=1).
- Label Noise in ML: The study highlights that using VIIRS data as ground truth for training machine learning models without accounting for misalignments introduces "label noise," potentially leading to sub-optimal classifiers.
- Future Directions: The authors suggest extending this methodology for burned area analysis, transitioning from pixel-level validation to event-level assessment for a more comprehensive understanding of fire dynamics. They also propose further investigation into the impact of misalignments on training machine learning models.
Summary updated