DeepSpecN: A new hybrid method combining PROSPECT-PRO and Conv-Transformer to estimate leaf nitrogen content from leaf reflectance

Shuai Yang, Anirudh Belwalkar, Dong Li, Yufeng Ge, Tao Cheng, Fei Wu, Litao Peng, Daoliang Li, Kang YuPlant 2025 · DOI

TL;DR

This paper introduces DeepSpecN, a novel hybrid deep learning method for accurately estimating leaf nitrogen content (LNC) from hyperspectral reflectance data. By combining a radiative transfer model (PROSPECT-PRO) with a Conv-Transformer architecture and a smart sample selection strategy, DeepSpecN achieves high accuracy without requiring in-situ training data, addressing a key challenge in remote sensing for agriculture.

What Problem Does This Paper Address?

Accurately measuring leaf nitrogen content (LNC) is vital for crop health monitoring and optimizing fertilizer use. Traditional lab methods are time-consuming and destructive. Remote sensing offers a non-destructive alternative, but existing methods have limitations:

  • Empirical methods: Rely on statistical relationships and require extensive in-situ data for training, making them prone to poor performance when applied to new environments or crop varieties (domain shift).
  • Physically-based methods: Use radiative transfer models (RTMs) but often suffer from ill-posed inversions, meaning different leaf properties can produce similar spectral signatures.
  • Hybrid methods: Combine the strengths of both but can still struggle with "domain shift" – the difference in data distribution between simulated RTM data and real-world measurements. This is exacerbated by issues like specular reflection in measured data and potential inaccuracies in simulated data.

This paper aims to overcome these limitations by developing a hybrid approach that can learn effectively from simulated data and generalize well to real-world measurements, even without direct in-situ training data.

Methods

DeepSpecN is a multi-component hybrid method:

  1. Continuous Wavelet Transform (CWT): Applied to hyperspectral reflectance data to reduce the impact of specular reflection, a common artifact in measured leaf spectra that can distort RTM-based simulations.
  2. PROSPECT-PRO: A radiative transfer model used to generate a large, simulated dataset of leaf reflectance spectra. This model is capable of estimating leaf protein content, which is then used to infer LNC.
  3. Conv-Transformer Architecture: A novel deep learning model that combines:
  4. 1D Convolutional Neural Network (1D-CNN): For extracting local features and capturing correlations between adjacent spectral bands.
  5. Transformer Encoder Blocks: For capturing long-range dependencies and global patterns within the hyperspectral data. This architecture is designed to handle high-dimensional spectral data effectively.
  6. Spectral Similarity-Based Sample Selection: A crucial component for mitigating domain shift. This method selects the most "representative" simulated samples from the large PROSPECT-PRO dataset that are spectrally similar to the target in-situ data. This ensures the training data is more relevant and reduces the burden of training on the entire massive simulated dataset.

The method essentially uses PROSPECT-PRO to create a rich simulated dataset, then employs CWT to harmonize spectral characteristics, and finally uses the Conv-Transformer with a carefully selected subset of simulated data to learn the LNC estimation task.

Datasets

  • Maize Leaf Reflectance Data: Bidirectional reflectance spectra from 1724 maize leaves were collected and used for validation. Leaf area, fresh weight, dry weight, and leaf nitrogen content (determined via Dumas method) were also measured.
  • Simulated Dataset: A large dataset of 200,000 simulated leaf reflectance spectra was generated using the PROSPECT-PRO model.
  • Validation Datasets: Datasets from wheat, rice, and sorghum were used to test the generalizability of the DeepSpecN model.

Key Insights and Ideas

  • The power of spectral similarity for domain adaptation: Simply using a large simulated dataset isn't enough. Selecting samples that are spectrally similar to real-world measurements is critical for bridging the gap between simulated and actual data.
  • CWT as a pre-processing step: CWT is effective in reducing the impact of specular reflection, a common issue in measured hyperspectral data that can hinder model performance.
  • Conv-Transformer excels at spectral feature extraction: The combination of 1D-CNN for local features and Transformer for global dependencies proves highly effective for complex hyperspectral data.
  • Chlorophyll-based LNC estimation is more robust: Empirical formulas derived from chlorophyll content generally yielded more accurate LNC estimates than those based on leaf protein content.
  • Generalizability across crops is achievable: The proposed method, particularly with the sample selection strategy, shows promise for application beyond maize to other crop species.

Key Results

  • DeepSpecN achieved the highest estimation accuracy among all tested methods, including physically-based, hybrid, and parametric regression approaches.
  • When trained on representative samples selected by spectral similarity, DeepSpecN achieved an RMSE of 0.247 g/m² and an R² of 0.665 for maize LNC estimation.
  • The spectral similarity-based sample selection strategy significantly improved performance compared to training on the entire simulated dataset.
  • Validation on wheat, rice, and sorghum confirmed the generalizability of DeepSpecN, with promising LNC prediction accuracy across these species.

Limitations and Future Work

  • Crop-specific calibration: While DeepSpecN shows generalizability, the accuracy can be influenced by the representativeness of the calibration dataset for specific crops. Future work should focus on collecting more diverse in-situ data to refine model parameters for various species.
  • LNC formulation impact: The study noted that LNC estimation based on protein-to-nitrogen conversion was less accurate, potentially due to weak protein absorption or crop-specific coefficient issues. Further research into optimal LNC formulations and crop-tailored coefficients is needed.
  • Scalability to canopy level: The current study focuses on leaf-level measurements. Future research should explore extending DeepSpecN to canopy-level nitrogen modeling and multiple-trait estimation.
  • Environmental factors: Investigating the impact of diverse environmental conditions on LNC estimation and model transferability is a crucial next step.

Summary updated

Abstract

Accurate, non-destructive quantification of leaf nitrogen content (LNC) is crucial for monitoring crop health and growth. Traditional empirical methods require extensive in-situ data for training, while physically-based methods are limited by ill-posed inversion, and hybrid methods suffer from domain shift between in-situ and simulated data. To overcome these limitations, this study introduces DeepSpecN, a novel hybrid method for maize LNC estimation using leaf-scale hyperspectral bidirectional reflectance. Without requiring in-situ data for training, DeepSpecN combines four key components: continuous wavelet transform (CWT) for reducing specular reflection, PROSPECT-PRO for simulating training data, an improved Transformer model for feature learning, and a spectral similarity-based sample selection method for selecting more valuable training samples. DeepSpecN and other methods, including physically-based methods, non-parametric regression based hybrid methods, and parametric regression methods based on vegetation indices (VIs), were validated using bidirectional reflectance data from 1724 maize leaves. The results showed that, when trained on representative samples, DeepSpecN achieved the highest estimation accuracy among all the methods (RMSE ​= ​0.247 ​g/m 2 , R 2 ​= ​0.665). The sample selection strategy mitigated the effects of domain shift by identifying representative training samples with high spectral similarity from the simulated database. Furthermore, the results showed that the Chlorophyll (Chl)-based empirical formulas estimated maize LNC more accurately than those based on leaf protein content. Moreover, the validation results on four different crop species confirm the generalizability of DeepSpecN. Our findings demonstrate the potential of hybrid methods that utilize bidirectional reflectance spectra, developed by addressing the domain shift issue, to improve the LNC estimation accuracy.
BibTeX
@inproceedings{yang2025deepspecn,
  author = {Shuai Yang and Anirudh Belwalkar and Dong Li and Yufeng Ge and Tao Cheng and Fei Wu and Litao Peng and Daoliang Li and Kang Yu and Daoliang Li and Kang Yu},
  title = {{DeepSpecN: A new hybrid method combining PROSPECT-PRO and Conv-Transformer to estimate leaf nitrogen content from leaf reflectance}},
  booktitle = {Plant},
  year = {2025},
  doi = {10.1016/j.plaphe.2025.100125},
}
Figure