Analytical-Chemistry-Informed Transformer for Infrared Spectra Modeling

Shiluo Huang, Yining Jin, Wei Jin, Ying MuAAAI 2025 · DOI

TL;DR

This paper introduces the Analytical-Chemistry-Informed Transformer (ACT), a novel deep learning architecture for infrared (IR) spectroscopy that integrates domain-specific knowledge to improve calibration transfer and general spectral analysis. ACT significantly reduces prediction errors, especially in challenging calibration transfer scenarios, by incorporating learnable spectral processing and a specialized spectral-attention mechanism.

What Problem Does This Paper Address?

Infrared (IR) spectroscopy is a powerful tool for chemical analysis, but its practical application faces a significant challenge known as "calibration transfer." This occurs when a model trained on IR spectra from one instrument or experimental setup performs poorly on spectra collected under different conditions. This degradation is due to variations in measurement environments, instruments, and sample preparation, which introduce domain shifts. Traditional methods often require retraining or access to target domain data, which can be costly and impractical. While deep learning (DL) has shown promise in other fields, its application to IR spectroscopy is still nascent, and existing DL models often lack the domain-specific inductive biases needed to handle these calibration transfer issues effectively. This paper aims to bridge this gap by developing a DL model that explicitly incorporates analytical chemistry knowledge to learn domain-invariant representations.

Methods

The proposed Analytical-Chemistry-Informed Transformer (ACT) builds upon the Transformer architecture but introduces two key novel modules informed by analytical chemistry principles:

  • Learnable Spectral Processing: This module integrates spectral pre-processing, tokenization, and post-processing. It features a reversible baseline corrector inspired by chemometrics. Unlike traditional methods that can "over-process" spectra and lose valuable information, this reversible approach allows the model to recover baseline information, which can contain discriminative features. The corrected spectra are then tokenized into patches, capturing local spectral shapes.
  • Spectral-Attention: This module replaces the standard self-attention in Transformers. It comprises two sub-modules:
  • Inter-spectral Attention: This mechanism uses a set of reference spectra from the training set to refine the attention map. These reference spectra act as a "memory" to help the model learn domain-invariant features by highlighting relevant spectral regions across different samples.
  • Intra-spectral Attention: This sub-module leverages a correlation matrix that captures relationships between spectral bands. This helps the model understand how different peaks within a spectrum (even non-adjacent ones) relate to each other, reflecting the underlying chemical compounds.

The final output of the spectral-attention module is a combination of these two attention mechanisms, further processed by a feed-forward network. For regression tasks, a trainable label projecting module is also included to map the model's output to the original label space.

Datasets

The paper evaluates ACT on 9 real-world IR spectral modeling tasks derived from 5 publicly available datasets:

  • Tablet Dataset: Contains NIR spectra of tablets collected by two different spectrometers, used for 4 calibration transfer tasks (e.g., Tablet(1,2) where training is from spectrometer 1 and testing from spectrometer 2).
  • Melamine Dataset: NIR spectra of melamine-formaldehyde with varying compositions, used for 2 turbidity point prediction tasks (MF(R562,R568) and MF(R568,R562)).
  • Mango_DMC Dataset: NIR spectra of intact mango fruit for predicting dry matter content.
  • Strawberry Dataset: MIR spectra of strawberry purees for classification.
  • Apple Leaf Dataset: NIR spectra of apple leaves for classification.

These datasets cover applications in pharmacy, chemistry, and agriculture, and include both calibration transfer and regular spectral analysis tasks.

Key Insights and Ideas

  • Domain Knowledge is Crucial for IR DL: The paper strongly advocates that simply applying standard DL architectures (like Transformers) to IR data is insufficient. Integrating domain-specific knowledge from analytical chemistry is key to overcoming challenges like calibration transfer.
  • Baselines Aren't Just Noise: The learnable spectral processing module demonstrates that baselines, often removed as noise, can contain valuable discriminative information. Reversible baseline correction allows ACT to leverage this information.
  • Attention Can Be Smarter: The spectral-attention mechanism, by incorporating reference spectra (inter-spectral) and spectral band correlations (intra-spectral), guides the model to focus on more relevant spectral features, mimicking how human experts analyze spectra.
  • Domain Invariance Achieved: ACT successfully learns domain-invariant representations, enabling it to perform well on unseen data from different spectral collection processes without requiring target domain data during training.

Key Results

  • Calibration Transfer Performance: ACT significantly outperforms existing deep learning methods on calibration transfer tasks, reducing the Root Mean Square Error of Prediction (RMSEP) by over 20% compared to other DL models. For Tablet tasks, it achieved a 27% reduction in RMSEP and a 23% reduction in MAE.
  • Competitive with Traditional Methods: ACT achieves comparable or better results than traditional calibration transfer methods (like di-PLS, JDOT, DIPALS) even when those methods are allowed to use unlabeled target domain data, while ACT does not.
  • Regular Spectral Analysis: ACT also performs competitively on regular IR spectral analysis tasks, achieving an average rank of 1.2 across regression and classification tasks, indicating its general effectiveness.
  • Efficiency: The computational cost of ACT's spectral-attention is manageable, with complexity related to the number of spectral bands, which is typically limited.

Limitations and Future Work

  • Reference Spectra Selection: The performance of inter-spectral attention can be influenced by the selection of reference spectra. While the paper shows robustness to random seed variations, further research into optimal reference selection strategies could be beneficial.
  • Hyperparameter Tuning: While ACT performs well, like any deep learning model, its performance can be sensitive to hyperparameter settings, which are tuned based on the calibrating sets.
  • Future Directions: The authors suggest that this work promotes the development of deep learning in IR spectroscopy by demonstrating the value of integrating domain knowledge. Future work could explore more sophisticated ways to incorporate chemometric principles and further refine the spectral-attention mechanism. The visualization of attention maps suggests that ACT learns representations aligned with expert judgment, which could be further explored for interpretability.

Summary updated

BibTeX
@inproceedings{huang2025analytical,
  author = {Shiluo Huang and Yining Jin and Wei Jin and Ying Mu},
  title = {{Analytical-Chemistry-Informed Transformer for Infrared Spectra Modeling}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33917},
}
Figure