A Two-Step Simulated Annealing Algorithm for Spectral Data Feature Extraction

Jian Pei, Liang Xu, Yitong Huang, Qingbin Jiao, Ming‐Yu Yang, Ding Ma, Sijia Jiang, Hui Li, Yuhang Li, Siqi Liu, Wěi Zhāng, Jiahang Zhang, Xin TanSensors 2023 · DOI

TL;DR

This paper introduces a novel "two-step simulated annealing algorithm" (TSSA) that combines global and local optimization to improve spectral data feature extraction. By integrating the Boruta algorithm for feature relevance assessment, TSSA significantly enhances modeling accuracy and stability for quantitative inversion tasks, outperforming traditional methods.

What Problem Does This Paper Address?

High-resolution spectral data, while rich in information, suffers from significant redundancy and the presence of irrelevant bands. This "curse of dimensionality" adversely affects the performance of modeling and quantitative inversion algorithms, leading to low accuracy and poor stability. Traditional feature extraction methods often struggle to effectively identify truly relevant spectral features, either by being too simplistic, prone to overfitting, or failing to adequately consider the correlation between extracted features and the target variable. This paper aims to tackle these limitations by developing a more robust and accurate feature extraction technique.

Methods

The core contribution of this paper is the development of the Two-Step Simulated Annealing Algorithm (TSSA). This algorithm builds upon the traditional Simulated Annealing (SA) algorithm by incorporating a "Boruta algorithm-based local optimization process."

Here's a breakdown:

  • Traditional SA: SA is a global optimization technique that explores the solution space by iteratively making random changes and accepting them based on a probability that depends on the temperature parameter. While good for global search, it can sometimes jump out of important regions too quickly and may select features weakly correlated with the target variable.
  • Boruta Algorithm: This algorithm is a wrapper method that uses Random Forests to assess feature importance. It works by comparing the importance of real features against the importance of "shadow" features (randomly permuted versions of the original features). This helps to identify features that are truly relevant and not just coincidentally correlated.
  • TSSA's Innovation: TSSA integrates the Boruta algorithm into the SA framework.
  • Step 1 (Global Optimization): This step is similar to traditional SA, using a probabilistic approach to explore the feature space.
  • Step 2 (Local Optimization): When the global optimization step identifies a potential solution, TSSA employs the Boruta algorithm to filter out irrelevant or weakly correlated features. These filtered-out features are then replaced with randomly selected, non-coinciding bands. This local optimization step refines the feature set, ensuring strong correlation with the dependent variable and reducing redundancy.

The TSSA algorithm is designed to overcome SA's tendency to get stuck in local optima and to ensure that selected features are strongly linked to the target variable.

Datasets

The study utilized spectral data collected from field measurements in two typical eutrophic shallow lakes in China: Chaohu Lake and Taihu Lake.

  • Data Collection: Spectral data was collected using a self-developed lake cyanobacterial biomass spectral-detection system. Water samples were also collected simultaneously for laboratory analysis of chlorophyll a (Chla) concentration.
  • Data Split: The collected 107 data samples were randomly divided into a training set (79 samples) and a validation set (28 samples).
  • Benchmark: While no new benchmark datasets were introduced, the study used these collected datasets to evaluate the performance of TSSA against several other feature extraction methods, including the correlation coefficient method, band ratio method, PCA, SPA, and the standard SA algorithm.

Key Insights and Ideas

  • The Synergy of Global and Local Optimization: The paper demonstrates that combining a global search strategy (SA) with a feature relevance-focused local refinement (Boruta) is crucial for effective spectral feature extraction. This two-step approach addresses the limitations of each method individually.
  • Boruta's Role in Feature Relevance: Integrating Boruta is key to ensuring that the selected spectral features are not just statistically significant but also meaningfully correlated with the target variable (Chla concentration in this case). This directly tackles the problem of including irrelevant information.
  • Improved Model Stability and Accuracy: By reducing data redundancy and focusing on relevant features, TSSA leads to more stable and accurate quantitative inversion models. This is critical for reliable environmental monitoring and applications.
  • Interpretability: Unlike some methods that transform data into abstract components (like PCA), TSSA's feature selection process aims to retain the interpretability of the original spectral bands.

Key Results

The experimental results highlight the superiority of the TSSA algorithm:

  • Modeling Accuracy: TSSA achieved the highest modeling accuracy among the six tested feature extraction methods.
  • Determination coefficient (R²) of 0.9654.
  • Root Mean Square Error (RMSE) of 3.6723 µg/L.
  • Mean Absolute Error (MAE) of 3.1461 µg/L.
  • Comparison: Compared to other methods, TSSA showed significant improvements:
  • R² increased by 0.1908–0.4623.
  • RMSE decreased by 5.7039–10.249 µg/L.
  • MAE decreased by 4.1575–7.6202 µg/L.
  • Validation Performance: The validation dataset analysis showed a strong correlation between predicted and measured values for TSSA, with the fitted line closely approximating the y=x line (y = 0.99x + 0.05).

Limitations and Future Work

  • Limitations:
    • The paper notes that traditional SA algorithms can have slow convergence for large-scale problems. While TSSA aims to improve this, the computational cost of the Boruta step might still be a factor for extremely large datasets.
  • The study focused on Chla concentration in shallow lakes. The applicability of TSSA to other types of spectral data or environments might require further investigation.
  • Future Work:
    • The authors suggest that this research provides new directions for spectral feature extraction and lays the foundation for accurate, rapid, and low-cost detection of Chla content. Future work could explore extending TSSA to other spectral analysis tasks and different environmental monitoring applications. The authors also provide a GitHub link for the core code of the local optimization process, encouraging further development and application.

Summary updated

Abstract

To address the shortcomings in many traditional spectral feature extraction algorithms in practical application of low modeling accuracy and poor stability, this paper introduces the "Boruta algorithm-based local optimization process" based on the traditional simulated annealing algorithm and proposes the "two-step simulated annealing algorithm (TSSA)". This algorithm combines global optimization and local optimization. The Boruta algorithm ensures that the feature extraction results are all strongly correlated with the dependent variable, reducing data redundancy. The accuracy and stability of the algorithm model are significantly improved. The experimental results show that compared with the traditional feature extraction method, the accuracy indexes of the inversion model established by using the TSSA algorithm for feature extraction were significantly improved, with the determination coefficient R<sup>2</sup> of 0.9654, the root mean square error (RMSE) of 3.6723 μg/L, and the mean absolute error (MAE) of 3.1461 μg/L.
BibTeX
@inproceedings{pei2023two,
  author = {Jian Pei and Liang Xu and Yitong Huang and Qingbin Jiao and Ming‐Yu Yang and Ding Ma and Sijia Jiang and Hui Li and Yuhang Li and Siqi Liu and Wěi Zhāng and Jiahang Zhang and Xin Tan},
  title = {{A Two-Step Simulated Annealing Algorithm for Spectral Data Feature Extraction}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23020893},
}
Figure