Evolving multispectral sensor configurations using genetic programming for estuary health monitoring

Mitchell Rogers, Mihailo Azhar, Stefano Schenone, Simon F. Thrush, Bing Xue, Mengjie Zhang, Patrice DelmasJournal of the Royal Society of New Zealand 2024 · DOI

TL;DR

This paper introduces a novel genetic programming (GP) approach to automatically select optimal wavelengths and extract features from hyperspectral images for monitoring estuary health. Their method, applied to sediment analysis, outperforms traditional wavelength selection techniques and offers a more flexible and interpretable way to build predictive models for environmental monitoring.

What Problem Does This Paper Address?

Assessing the health of ecosystems like estuaries is vital for effective management and decision-making. Hyperspectral imaging offers a powerful, non-invasive way to gather detailed spectral information from these environments. However, hyperspectral images are incredibly data-rich (high-dimensional), making them computationally expensive to process and analyze. Real-world applications often require simpler, lower-cost multispectral sensors, which necessitates selecting only the most informative wavelengths from the vast hyperspectral data. Existing methods often treat wavelength selection as a simple optimization problem, ignoring the complex relationships between adjacent wavelengths and the potential for extracting informative spatial features alongside spectral ones. This paper aims to bridge this gap by developing a more sophisticated method that can intelligently select wavelengths and extract relevant features, even from limited datasets.

Methods

The core of this work is a novel application of Genetic Programming (GP). GP is an evolutionary computation technique that evolves computer programs (represented as trees) to solve problems. The authors developed two GP program structures:

  1. Spectra-based GP: This approach takes preprocessed reflectance spectra from hyperspectral images and evolves programs to select informative wavelengths and combine them into a feature vector.
  2. Image-based GP: This more complex structure extends the spectra-based approach by also incorporating spatial information. It can extract features from individual spectral bands (as grayscale images) and combine them with spectral features. This allows for the extraction of texture features (like GLCM and Gabor filters) alongside spectral information.

Both structures use a function set including interval selection, spectral/image processing, aggregation, and feature concatenation. The evolved GP trees generate feature vectors that are then used to train Support Vector Regression (SVR) models to predict sediment properties. The GP process is driven by fitness functions (R² and Mean Squared Error) and evolutionary operators (selection, crossover, mutation).

Datasets

The study utilized a newly collected dataset comprising:

  • Hyperspectral images of sediment: Acquired in Whangateau Harbour, New Zealand, using a portable SPECIM IQ camera.
  • Ground-truth measurements: Corresponding measurements of sediment porosity and organic matter content for 30 sampling locations.

This dataset, collected in situ, is a key contribution, allowing for the direct application and validation of their methods in a real-world estuarine environment.

Key Insights and Ideas

  • GP as a Unified Feature Engineering Tool: GP excels at automating complex feature engineering pipelines, simultaneously handling wavelength selection, spectral processing, and even spatial feature extraction. This reduces the need for extensive domain expertise in designing these steps.
  • Spectra vs. Image Features: For this specific task (sediment analysis), the spectra-based GP approach generally outperformed the image-based approach. This suggests that spectral information was more critical than spatial texture for predicting sediment properties in this context, or that the larger search space of the image-based approach made finding optimal solutions more challenging.
  • Interpretability of GP: The tree-based structure of GP solutions offers a degree of interpretability, allowing researchers to understand which functions and wavelengths were deemed important by the evolved programs.
  • Potential for Noisy Correlations: The authors observed rare instances where GP models showed a significant drop in performance on unseen data, suggesting they might find noisy correlations specific to the training set. This highlights the importance of robust validation, especially with limited datasets.

Key Results

  • Competitive Performance: The proposed spectra-based GP solutions achieved competitive results compared to established wavelength selection methods like SPA, CARS, and RC, often selecting fewer wavelengths while maintaining or improving predictive performance.
  • Organic Matter Prediction: The best-evolved spectra-based GP model, when used with a linear SVR, achieved an R² of 0.58 for predicting organic matter content across all collected images.
  • Porosity Prediction Challenges: Full-wavelength models and most wavelength selection methods struggled to reliably predict sediment porosity. The authors noted that porosity was not considered in further wavelength selection experiments due to these poor results.
  • GP Outperforms Others: Across various metrics, GP-based methods generally outperformed the comparison wavelength selection methods on the prediction set.

Limitations and Future Work

  • Dataset Size: The authors acknowledge that the relatively small dataset size may have contributed to some models finding noisy correlations that didn't generalize well.
  • Porosity Prediction: The inability of the models to reliably predict porosity is a significant limitation for environmental monitoring.
  • Fitness Function Choice: While both R² and MSE were used, the paper doesn't definitively conclude which is superior for this task, though MSE might offer a better error surface for optimization.

Future work suggested includes:

  • Data Augmentation and Transfer Learning: To address the limitations of small datasets and improve generalization.
  • Few-Shot Learning: To further enhance performance on limited data scenarios.
  • Exploring Other Applications: Investigating scenarios where texture features might be more informative, potentially making the image-based GP approach more advantageous.
  • Manual Pruning: Further reducing the number of selected wavelengths by identifying those with minimal impact on prediction.

Summary updated

Abstract

Assessing ecosystem health on a large scale is crucial for a wide range of management and regulatory decisions. Technologies such as hyperspectral imaging allow noninvasive and rapid estimation of key attributes based on observed reflectance. However, these images are high-dimensional and real-world applications require models based on fewer wavelengths. This paper proposes a new wavelength selection and feature extraction method for hyperspectral image analysis based on genetic programming to automatically select key wavelength regions and informative image features. A dataset of hyperspectral images of sediment in the field was collected and paired with ground-truth measurements of the sediment porosity and organic matter content. Two new program structures were proposed to construct feature extraction trees from either the mean reflectance spectra (spectra-based) or full hyperspectral images (image-based). SVR models were constructed to predict attributes based on the extracted features. Various regression models were used to predict the porosity and organic matter content. Full-wavelength models were constructed to reliably predict the organic matter content. The proposed spectra-based genetic programming solutions show competitive results compared to common wavelength selection methods, such as SPA, CARS, and RC. Finally, the best-evolved solution was applied to predict sediment organic matter content across all collected images.
BibTeX
@article{rogers2024evolving,
  author = {Mitchell Rogers and Mihailo Azhar and Stefano Schenone and Simon F. Thrush and Bing Xue and Mengjie Zhang and Patrice Delmas},
  title = {{Evolving multispectral sensor configurations using genetic programming for estuary health monitoring}},
  journal = {Journal of the Royal Society of New Zealand},
  year = {2024},
  doi = {10.1080/03036758.2024.2393297},
}
Figure