Deep Remote Sensing Methods for Methane Detection in Overhead Hyperspectral Imagery

Satish Kumar, Carlos Torres, Oytun Ulutan, Alana K. Ayasse, Dar A. Roberts, B. S. ManjunathWACV 2020 · OpenReview

TL;DR

This paper introduces H-mrcnn, a novel deep learning approach for detecting and delineating methane (CH4) plumes in hyperspectral imagery. By combining optimized match-filtering with an ensemble of Mask R-CNN detectors, H-mrcnn significantly reduces manual annotation effort and processing time, achieving expert-level accuracy in identifying these critical greenhouse gas emissions.

What Problem Does This Paper Address?

Detecting and monitoring methane (CH4) emissions is crucial for understanding and mitigating climate change. CH4 is a potent greenhouse gas, and its release often originates from diffuse sources like leaks in oil and gas infrastructure. Hyperspectral imagery, which captures detailed spectral information across many narrow bands, is a powerful tool for this task. However, existing methods for CH4 detection in hyperspectral data face significant challenges:

  • Manual Labor and Scalability: Current approaches often require extensive manual inspection and annotation by domain experts, making them slow, expensive, and difficult to scale to large geographic areas or frequent monitoring.
  • Small Pixel Footprint: CH4 plumes, especially from diffuse sources, can have a very small signature relative to the overall image area, making them hard to distinguish from background noise and terrain variations.
  • Human Error: Manual annotation is prone to errors and inconsistencies.
  • Computational Complexity: Processing vast amounts of hyperspectral data can be computationally intensive.

This paper aims to overcome these limitations by developing an automated, efficient, and accurate method for CH4 plume detection and delineation.

Methods

The proposed H-mrcnn (Hyperspectral Mask-RCNN) method is a hybrid approach that integrates principled statistics, signal processing, and deep neural networks. The core components are:

  • Optimized Match-Filtering: The raw hyperspectral data (432 bands) is first processed using an optimized match-filtering technique. This step aims to extract relevant spectral information related to CH4 absorption bands (around 2100-2400nm) while also capturing terrain information from visible and near-infrared bands. This process is designed to be computationally efficient, reducing processing time significantly compared to traditional methods.
  • Sliding Window and Feature Extraction: The output of the match-filtering is then processed using a sliding window approach. This extracts spatial and spectral features from the data, creating input "tiles" for the deep learning models. These tiles are composed of three channels: locally normalized CH4 information, globally normalized CH4 information, and greyscale terrain information.
  • Ensemble of Mask R-CNN Detectors: A key innovation is the use of an ensemble of Mask R-CNN detectors. Each detector in the ensemble is trained on specific sets of spectral bands. This allows the system to learn different aspects of the CH4 signature and terrain variations. For instance, detectors trained on visible bands can learn to distinguish terrain features, while those trained on short-infrared bands focus on the CH4 absorption signal. This ensemble approach helps in learning correlations between pixels and improving plume detection.
  • Decision Fusion: The outputs (predicted masks) from the individual ensemble detectors are then fused using a simple 2-layer perceptron network. This network learns to assign weights to each detector's output, effectively creating a weighted average that produces a final, refined plume mask. This fusion step helps to improve accuracy and reduce false positives by combining the strengths of multiple specialized detectors.

Datasets

The research utilizes two datasets derived from the Airborne Visible/Infrared Imaging Spectrometer Next Generation (AVIRIS-NG) instrument:

  • Dataset A (XA): This is a pre-processed, 4-band dataset where the original 432 bands have been condensed using conventional match-filtering techniques. It includes bands for red, green, blue reflectance, and a fourth band representing CH4 concentration in ppm×m. This dataset was used for initial validation and tuning of a naive single-band detector.
  • Dataset B (XB): This is the raw, unrectified, 432-band dataset. It covers a wide spectral range (380nm to 2510nm) and captures diverse terrain types, including plains, mountains, vegetation, and industrial areas, which can contain potential "confusers" (e.g., paints with similar spectral signatures to CH4). This dataset was used for the development and evaluation of the full H-mrcnn system.

The paper does not explicitly introduce a new benchmark dataset but uses existing AVIRIS-NG data to evaluate its proposed method.

Key Insights and Ideas

  • Ensemble Learning for Hyperspectral Data: The paper demonstrates that an ensemble of specialized deep learning detectors, each tuned to different spectral subsets, can outperform a single monolithic detector. This allows the model to leverage the rich, multi-dimensional spectral information more effectively.
  • Fusion of Weak Detectors: Combining the outputs of multiple "weak" detectors (each potentially imperfect) through a learned fusion mechanism can lead to a robust and accurate final prediction. This is a powerful strategy for complex detection tasks.
  • Leveraging Terrain Information: Explicitly incorporating terrain information (via greyscale images) alongside spectral CH4 signals helps the model learn to distinguish actual plumes from background features and potential confusers.
  • Optimized Match-Filtering as Preprocessing: Using an efficient match-filtering step as a preprocessing stage significantly reduces the dimensionality and processing time of the raw hyperspectral data, making it amenable to deep learning without losing critical information.

Key Results

  • High Accuracy: The H-mrcnn method achieves an Intersection-over-Union (IOU) score of 87.95%, which is a significant improvement over the naive single-band detector (83.2%) and substantially better than classical machine learning methods. This metric indicates how well the predicted plume masks align with expert-generated ground truth.
  • Reduced Processing Time: The proposed method drastically reduces manual data processing and labeling time by a factor of ×12. The overall processing time for a dataset is reduced from over 3 hours to approximately 22 minutes (including match-filtering and detection).
  • Improved Detection: The ensemble detector is capable of eliminating more false positives compared to classical methods and even the naive Mask R-CNN, leading to more reliable CH4 plume detection.
  • Scalability: The approach is designed to scale to larger datasets, addressing a key limitation of manual annotation methods.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the naive Mask-RCNN detector, while effective, is limited by the number of training samples.
  • The performance of the ensemble is sensitive to the choice of spectral band subsets and window sizes.
  • While the method is generalizable, it is demonstrated specifically for CH4.
  • Future Work:
    • Extension to Other Gases: Applying the H-mrcnn framework to detect other atmospheric gases.
  • Online Learning: Developing online learning capabilities for real-time tuning of detectors, especially for point sources and diffused sources.
  • Time-Series Analysis: Investigating plume dispersion, gas flux, life-span, and evolution over time using sequential data.
  • Further Optimization: Exploring more advanced ensemble fusion techniques and network architectures.

Summary updated

Abstract

Effective analysis of hyperspectral imagery is essential for gathering fast and actionable information of large areas affected by atmospheric and green house gases. Existing methods, which process hyperspectral data to detect amorphous gases such as CH <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</inf> require manual inspection from domain experts and annotation of massive datasets. These methods do not scale well and are prone to human errors due to the plumes’ small pixel-footprint signature. The proposed Hyperspectral Mask-RCNN (H-mrcnn) uses principled statistics, signal processing, and deep neural networks to address these limitations. H-mrcnn introduces fast algorithms to analyze large-area hyper-spectral information and methods to autonomously represent and detect CH <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</inf> plumes. H-mrcnn processes information by match-filtering sliding windows of hyperspectral data across the spectral bands. This process produces information-rich features that are both effective plume representations and gas concentration analogs. The optimized matched-filtering stage processes spectral data, which is spatially sampled to train an ensemble of gas detectors. The ensemble outputs are fused to estimate a natural and accurate plume mask. Thorough evaluation demonstrates that H-mrcnn matches the manual and experience-dependent annotation process of experts by 85% (IOU). H-mrcnn scales to larger datasets, reduces the manual data processing and labeling time (×12), and produces rapid actionable information about gas plumes.
BibTeX
@inproceedings{kumar2020deep,
  author = {Satish Kumar and Carlos Torres and Oytun Ulutan and Alana K. Ayasse and Dar A. Roberts and B. S. Manjunath},
  title = {{Deep Remote Sensing Methods for Methane Detection in Overhead Hyperspectral Imagery}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=xlhx1c-oGu},
}
Figure