A Raman spectroscopy algorithm based on convolutional neural networks and multilayer perceptrons: qualitative and quantitative analyses of chemical warfare agent simulants

Jie Wu, Fei Li, Jingwen Zhou, Hongmei Li, Zilong Wang, Xian-Ming Guo, Yue‐Jiao Zhang, Lin Zhang, Pei Liang, Shisheng Zheng, Jian‐Feng LiThe Analyst 2025 · DOI

TL;DR

Researchers have developed RS-MLP, a deep learning framework that combines convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) to identify and quantify chemical warfare agent (CWA) simulants in complex mixtures. By leveraging a multi-head attention mechanism and GAN-based data augmentation, the model achieves 100% identification accuracy and a concentration prediction error (RMSE) of less than 0.473%, even when spectra are highly overlapping.

What Problem Does This Paper Address?

Detecting persistent chemical warfare agents on-site is a high-stakes challenge for military and counter-terrorism teams. Raman spectroscopy is the "gold standard" for non-destructive, non-contact detection because it provides a unique "molecular fingerprint." However, real-world samples are rarely pure; they are messy mixtures.

Traditional methods like Partial Least Squares Regression (PLSR) struggle when:

  1. Spectra Overlap: Different chemicals have peaks in the same spots, making them hard to distinguish.
  2. Trace Components: Low-concentration chemicals are often drowned out by dominant ones or background noise.
  3. Non-linear Interactions: Chemicals in a mixture don't always "add up" linearly, confusing simpler algorithms.

Existing deep learning models often act as "black boxes," lacking the interpretability needed for high-stakes decision-making and requiring massive amounts of labeled data that are difficult to obtain for hazardous materials.

Methods

The authors propose the RS-MLP framework, which consists of three main stages:

  1. Pure Substance Feature Extraction: The system builds a reference library of "key peaks" (position, intensity, width) from pure simulants (DMMP, DIMP, and TEP).
  2. WGAN-GP Data Augmentation: To solve the data scarcity problem, the team used a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP). This allowed them to synthesize thousands of realistic Raman spectra for binary and ternary mixtures across a full gradient of concentrations, including "trace" scenarios (0.5% concentration).
  3. The RS-MLP Analytical Network:
    • Feature Extraction: Uses dilated convolutions to capture both global spectral patterns and local peak details.
  4. Multi-Head Attention: This is the "secret sauce." One set of attention heads focuses on Raman shift (position) to identify what is there, while another focuses on peak intensity to determine how much is there.
  5. MLP-Mixer: Performs hierarchical matching between the mixture features and the reference library.
  6. Multi-Task Learning: The model simultaneously predicts component presence, concentration, and an "uncertainty estimation" to provide a confidence score for its predictions.

Datasets

  • Primary Simulant Dataset: Focused on three common CWA simulants: DMMP (sarin simulant), TEP (VX simulant), and DIMP (nerve agent simulant). It included 21 binary and 24 ternary mixture combinations.
  • Hazardous Chemical Dataset: A larger, self-constructed validation set containing 75 substances, including 15 pure substances and various mixtures up to "quinary" (5-component) levels. This was used to test the model's generalization and robustness against baseline noise.

Key Insights and Ideas

  • Dual-Channel Matching: The most clever aspect of RS-MLP is how it decouples structural identification from quantification. By encoding position and intensity separately, the model mimics how a human expert looks at a spectrum—first identifying the "fingerprint" peaks and then looking at their heights.
  • GANs for "Filling the Gaps": Instead of just using simple linear mixing to create training data, the WGAN-GP learns the underlying distribution of Raman noise and non-linear interactions, making the synthetic training data much more representative of real-world sensor output.
  • Interpretability via Heatmaps: The authors used attention heatmaps to show exactly which spectral regions the model prioritized. This moves away from "black box" AI, allowing researchers to verify that the model is actually looking at chemically relevant peaks rather than background noise.
  • Dynamic Weighting for Trace Detection: The loss function was designed to penalize errors in low-concentration components more heavily. This prevents the model from ignoring trace chemicals that might be overshadowed by major components.

Key Results

  • Perfect Identification: The model achieved a 100% recognition rate for the four types of mixture combinations used in validation.
  • High Quantitative Precision: The average Root Mean Square Error (RMSE) for concentration prediction was less than 0.473%.
  • Superiority over Benchmarks: In head-to-head comparisons using the 75-substance hazardous dataset, RS-MLP significantly outperformed standard CNNs, ResNets, and LSTM-CNN hybrids. For complex quinary mixtures, RS-MLP maintained high accuracy while other models failed to recognize components (indicated by "F" in their results table).
  • Robustness: The model maintained its performance even when random baseline noise was introduced, proving its readiness for "dirty" field environments.

Limitations and Future Work

While the results are impressive, the authors note a few areas for improvement:

  • Simulation vs. Reality: There is still a slight performance gap when moving from purely simulated/GAN-generated data to real-world mixtures, likely due to complex environmental factors not fully captured in the simulation.
  • Quinary Complexity: While it outperformed other models, quinary (5-component) mixtures remain the most challenging scenario, with slightly lower $R^2$ values compared to simpler mixtures.
  • Future Directions: The authors suggest optimizing data augmentation strategies further and refining the network architecture to handle even more complex environmental matrices, such as chemicals contaminated on soil surfaces.

Summary updated

Abstract

Rapid and reliable detection of chemical warfare agents (CWAs) is essential for military defense and counter-terrorism operations. Although Raman spectroscopy provides a non-destructive method for on-site detection, existing methods show difficulty in coping with complex spectral overlap and concentration changes when analyzing mixtures containing trace components and highly complex mixtures. Based on the idea of convolutional neural networks and multi-layer perceptrons, this study proposes a qualitative and quantitative analysis algorithm of Raman spectroscopy based on deep learning (RS-MLP). The reference feature library is built from pure substance spectral features, while multi-head attention adaptively captures mixture weights. The MLP-Mixer then performs hierarchical feature matching for qualitative identification and quantitative analysis. The recognition rate of spectral data for the four types of combinations used for validation reached 100%, with an average root mean square error (RMSE) of less than 0.473% for the concentration prediction of three components. Furthermore, the model exhibited robust performance even under conditions of highly overlapping spectra. At the same time, the interpretability of the model is also enhanced. The model has excellent accuracy and robustness in component identification and concentration identification in complex mixtures and provides a practical solution for rapid and non-contact detection of persistent chemicals in complex environments.
BibTeX
@inproceedings{wu2025raman,
  author = {Jie Wu and Fei Li and Jingwen Zhou and Hongmei Li and Zilong Wang and Xian-Ming Guo and Yue‐Jiao Zhang and Lin Zhang and Pei Liang and Shisheng Zheng and Jian‐Feng Li},
  title = {{A Raman spectroscopy algorithm based on convolutional neural networks and multilayer perceptrons: qualitative and quantitative analyses of chemical warfare agent simulants}},
  booktitle = {The Analyst},
  year = {2025},
  doi = {10.1039/d5an00075k},
}
Figure