Stochastic Gradient Based Extreme Learning Machines For Online Learning of Advanced Combustion Engines

Vijay Manikandan Janakiraman, XuanLong Nguyen, Dennis AssanisarXiv 2015 · OpenReview

Abstract

In this article, a stochastic gradient based online learning algorithm for Extreme Learning Machines (ELM) is developed (SG-ELM). A stability criterion based on Lyapunov approach is used to prove both asymptotic stability of estimation error and stability in the estimated parameters suitable for identification of nonlinear dynamic systems. The developed algorithm not only guarantees stability, but also reduces the computational demand compared to the OS-ELM approach based on recursive least squares. In order to demonstrate the effectiveness of the algorithm on a real-world scenario, an advanced combustion engine identification problem is considered. The algorithm is applied to two case studies: An online regression learning for system identification of a Homogeneous Charge Compression Ignition (HCCI) Engine and an online classification learning (with class imbalance) for identifying the dynamic operating envelope of the HCCI Engine. The results indicate that the accuracy of the proposed SG-ELM is comparable to that of the state-of-the-art but adds stability and a reduction in computational effort.

Introduction

Homogeneous Charge Compression Ignition (HCCI) engines represent a promising technology for achieving high thermal efficiency alongside significantly reduced NOx and particulate emissions. However, the widespread adoption of HCCI is hindered by its complex, highly nonlinear combustion dynamics and a narrow stable operating window. Slight deviations in operating conditions can lead to undesirable phenomena such as misfire or knock. Effective control of HCCI engines requires accurate predictive models that can adapt in real-time to changing environmental conditions and engine wear.

Traditional offline models often fail to generalize to real-world scenarios. Consequently, online learning—where models are updated continuously using streaming sensor data—is critical. While Online Sequential Extreme Learning Machines (OS-ELM) using Recursive Least Squares (RLS) have been used, they often suffer from parameter instability and high computational costs. This paper proposes a Stochastic Gradient-based ELM (SG-ELM) to address these challenges.

Methodology: SG-ELM

The proposed SG-ELM leverages the architecture of Extreme Learning Machines, where hidden layer weights are randomly assigned and only output weights are learned. Unlike OS-ELM, which uses RLS, SG-ELM employs Stochastic Gradient Descent (SGD) for weight updates. This shift provides several advantages:

  1. Mathematical Stability: Using Lyapunov stability theory, the authors prove that the parameter estimation error in SG-ELM is asymptotically stable and bounded, preventing the "parameter explosion" seen in RLS-based methods.
  2. Computational Efficiency: By updating weights point-by-point without the need for complex covariance matrix inversions, SG-ELM significantly reduces the computational burden.
  3. Imbalance Handling: For classification tasks like misfire detection, a cost-sensitive SGD update rule is implemented to prioritize the rare but critical "unstable" states.

Experimental Setup

The algorithm was validated using experimental data from a 2.0L 4-stroke gasoline HCCI engine. The engine was equipped with a suite of sensors to monitor critical parameters such as intake/exhaust manifold pressures and temperatures, fuel injection timing, and in-cylinder pressure.

Schematic of the experimental HCCI engine setup, detailing the placement of various sensors for online monitoring.

The dataset comprised approximately 11,000 cycles for regression tasks (predicting NMEP and CA50) and over 20,000 cycles for classification tasks (identifying the operating envelope).

Results and Discussion

The SG-ELM demonstrated superior performance in both stability and speed. In regression tasks, it achieved a multi-step ahead RMSE of 0.0939, outperforming the OS-ELM (0.1024). More importantly, the norm of the estimated parameters remained low (3.71 for SG-ELM vs. 16.64 for OS-ELM), indicating a more robust and regularized model.

Experimental time-series data illustrating the highly nonlinear and transient nature of HCCI engine parameters across different operating cycles.

In classification, SG-ELM achieved a total accuracy of 87.92% and was particularly effective at identifying the minority class of unstable cycles, with a True Positive Rate of 98.76%. Computationally, SG-ELM was up to 4.6 times faster than OS-ELM, making it ideal for real-time implementation in Engine Control Units (ECUs).

Conclusion

The SG-ELM provides a stable, efficient, and accurate framework for the online identification of advanced combustion engines. By replacing RLS with a mathematically grounded SGD approach, the authors have addressed the critical issues of parameter instability and computational overhead. Future work will focus on integrating these predictive models into closed-loop control systems and testing on a wider variety of dynamic datasets.

Summary updated

Abstract

In this article, a stochastic gradient based online learning algorithm for Extreme Learning Machines (ELM) is developed (SG-ELM). A stability criterion based on Lyapunov approach is used to prove both asymptotic stability of estimation error and stability in the estimated parameters suitable for identification of nonlinear dynamic systems. The developed algorithm not only guarantees stability, but also reduces the computational demand compared to the OS-ELM approach based on recursive least squares. In order to demonstrate the effectiveness of the algorithm on a real-world scenario, an advanced combustion engine identification problem is considered. The algorithm is applied to two case studies: An online regression learning for system identification of a Homogeneous Charge Compression Ignition (HCCI) Engine and an online classification learning (with class imbalance) for identifying the dynamic operating envelope of the HCCI Engine. The results indicate that the accuracy of the proposed SG-ELM is comparable to that of the state-of-the-art but adds stability and a reduction in computational effort.
BibTeX
@misc{janakiraman2015stochastic,
  author = {Vijay Manikandan Janakiraman and XuanLong Nguyen and Dennis Assanis},
  title = {{Stochastic Gradient Based Extreme Learning Machines For Online Learning of Advanced Combustion Engines}},
  year = {2015},
  eprint = {1501.03975},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1501.03975},
}
Figure