Adaptive Deep Learning Modeling of Green Ammonia Production Process Based on Two-Layer Attention Mechanism LSTM

Jie Yang, Ji Zhao, Zhongbo Hu, Junxiang Wang, Xiaochuan Huang, Xu Ji, Ge HeProcesses 2025 · DOI

TL;DR

This paper introduces a novel "dual-layer attention LSTM" (DA-LSTM) model to accurately predict ammonia yield and key process indicators in green ammonia production. By employing two attention mechanisms, the model effectively handles the complex, nonlinear, and multi-timescale dynamics inherent in these systems, leading to improved prediction accuracy and interpretability.

What Problem Does This Paper Address?

Green ammonia production, a crucial pathway for decarbonization and energy transition, faces significant modeling challenges. The process involves tightly coupled units (renewable energy, electrolysis, Haber-Bosch synthesis) that generate data characterized by nonlinearity, uncertainty, noise, and multi-timescale dynamics. These complexities make it difficult to accurately predict ammonia yields and optimize production efficiency while controlling carbon emissions. Traditional modeling approaches struggle to capture these intricate interactions, hindering real-time optimization and control.

Methods

The core innovation is a Dual-Stage Attention-Based LSTM (DA-LSTM) architecture. This model builds upon the Long Short-Term Memory (LSTM) network, known for its ability to handle sequential data. The DA-LSTM introduces two sequential attention mechanisms:

  • Input Attention Mechanism (First Layer): This layer acts as a filter, adaptively weighting input features (process indicators) to screen out redundant or noisy variables. This helps in identifying the most critical parameters influencing the output.
  • Temporal Attention Mechanism (Second Layer): This layer focuses on the temporal aspect, dynamically assigning weights to different time steps within the input sequence. It captures time-varying feature weights, allowing the model to adaptively analyze the contribution of different sub-windows of data to the output over time.

This dual-layer approach aims to overcome the limitations of single-attention mechanisms, which struggle with feature redundancy, noise sensitivity, multi-timescale coupling, and interpretability in complex systems like green ammonia production.

Datasets

The model was implemented and validated on data collected from a dynamic simulation model of a renewable energy-coupled green ammonia demonstration project, built using Honeywell UniSim R460.1@ software. The dataset includes operational data spanning 90-50% of production capacity, with 56 selected process tags covering reaction kinetics, energy efficiency, material balance, and equipment status. The data was chronologically partitioned into training, validation, and test sets at an 8:1:1 ratio.

Key Insights and Ideas

  • Dual Attention for Enhanced Feature and Temporal Understanding: The combination of input and temporal attention is crucial. The input attention filters out noise and selects relevant features, while the temporal attention captures the dynamic importance of different time points, effectively addressing the multi-timescale nature of the process.
  • Improved Interpretability through Weight Visualization: The attention weights provide a visual heatmap, allowing engineers to intuitively understand which time steps and input features have the most significant impact on ammonia yield and temperature. This is invaluable for process optimization and identifying critical operational phases.
  • Mitigating Noise and Redundancy: The input attention mechanism is particularly effective at filtering out noise and redundant sensor data, a common issue in industrial settings, leading to more robust predictions.
  • Capturing Complex Dynamics: The DA-LSTM's ability to adaptively weight inputs and time steps allows it to better model the intricate, non-linear relationships and cross-scale interactions present in green ammonia production.

Key Results

The DA-LSTM model demonstrated superior performance compared to baseline models (Random Forest, SVM, ANN, and conventional LSTM) across key evaluation metrics:

  • Ammonia Production:
    • RMSE: DA-LSTM achieved 0.084, significantly lower than LSTM (0.113).
  • MAE: DA-LSTM achieved 0.044, lower than LSTM (0.065).
  • MAPE: DA-LSTM achieved 2.740, lower than LSTM (3.186).
  • First Bed Exit Temperature:
    • RMSE: DA-LSTM achieved 0.026, lower than LSTM (0.038).
  • MAE: DA-LSTM achieved 0.022, lower than LSTM (0.036).
  • MAPE: DA-LSTM achieved 0.086, significantly lower than LSTM (0.183).

The model also showed remarkable robustness across different data partitioning ratios and superior stability in blocked k-fold cross-validation compared to the baseline LSTM.

Limitations and Future Work

The authors acknowledge two main limitations:

  • Computational Demands: The DA-LSTM model exhibits higher computational demands and longer training times compared to conventional LSTM. This necessitates further research into enhancing its agility and computational efficiency.
  • Validation Scope: While validated on a simulation platform, the model requires additional validation using open-source benchmark datasets and real-world industrial data to comprehensively verify its efficacy.

Future work could focus on optimizing the model for faster inference and exploring its application in fault detection and real-time control systems, potentially integrating it with digital twin platforms. The interpretability offered by the attention mechanism also opens avenues for deeper insights into process anomalies and control strategies.

Summary updated

Abstract

Green ammonia, as a zero-carbon energy carrier, has emerged as a core process for achieving energy transition and chemical industry decarbonization through renewable energy-powered electrolytic hydrogen production integrated with low-carbon Haber–Bosch ammonia synthesis. However, the strong coupling among multiple units in green ammonia production systems, combined with operational data characteristics of nonlinearity, uncertainty, noise interference, and multi-timescale dynamics, creates significant challenges in accurately predicting ammonia yields and key process indicators, ultimately hindering online process parameter optimization and restricting improvements in production efficiency with effective carbon emission control. To address this, this study proposes a dual-layer attention LSTM model. The architecture constructs two sequential attention mechanisms: the first layer being an input attention mechanism for screening critical process indicators, followed by the second layer as a temporal attention mechanism that dynamically captures time-varying feature weights, enabling the adaptive analysis of sub-window contribution discrepancies to output variables across multiple time steps. Furthermore, the model is implemented and validated on a simulation platform of a renewable energy-coupled green ammonia demonstration project, with comparative analyses conducted against conventional LSTM and other baseline models. Experimental results demonstrate that the proposed model effectively adapts to complex scenarios in green ammonia production, including fluctuating renewable energy inputs and time-varying reaction conditions, providing reliable support for yield prediction and energy efficiency optimization. The developed methodology not only provides a novel approach for intelligent modeling of green ammonia production systems but also establishes a technical foundation for digital twin-based real-time control and dynamic scheduling research.
BibTeX
@inproceedings{yang2025adaptive,
  author = {Jie Yang and Ji Zhao and Zhongbo Hu and Junxiang Wang and Xiaochuan Huang and Xu Ji and Ge He},
  title = {{Adaptive Deep Learning Modeling of Green Ammonia Production Process Based on Two-Layer Attention Mechanism LSTM}},
  booktitle = {Processes},
  year = {2025},
  doi = {10.3390/pr13051480},
}
Figure