TL;DR
This paper introduces a novel "Student's-t Mixture Regression" (SMR) approach for developing robust soft sensors in industrial processes that operate in multiple modes and are prone to noisy data (outliers). SMR effectively handles these challenges by using a more robust statistical distribution than traditional methods, leading to more accurate predictions.
What Problem Does This Paper Address?
Many industrial processes aren't static; they operate under various conditions or "modes" (e.g., different production phases, feedstock changes). This multimode nature makes it difficult to build a single, accurate predictive model. Furthermore, industrial data is often "dirty" – it contains outliers due to faulty sensors, incorrect recording, or data entry errors. These outliers are hard to detect and remove, and they can significantly degrade the performance of predictive models, including soft sensors. Soft sensors are crucial for real-time monitoring and control of quality variables that are expensive or time-consuming to measure directly. The challenge is to create soft sensors that are both accurate in multimode environments and resilient to noisy data.
Methods
The core innovation is the Student's-t Mixture Regression (SMR). Unlike traditional Gaussian Mixture Models (GMMs) that assume data follows a normal distribution, SMR uses the Student's-t distribution. This distribution has "heavier tails," meaning it's less sensitive to extreme values (outliers).
Here's a breakdown of the SMR approach:
- Multimode Modeling: SMR models the process as a mixture of components, where each component represents a different mode or operating condition.
- Robustness via Student's-t: For each component, instead of a Gaussian distribution, a Student's-t distribution is used to model the secondary (easily measured) variables. This explicitly accounts for potential outliers.
- Explicit Functional Relationship: SMR directly models the relationship between the secondary variables and the primary (hard-to-measure quality) variables.
- Parameter Learning: An efficient Expectation-Maximization (EM) algorithm is developed to learn the parameters of the SMR model. This algorithm iteratively refines the model by estimating the probability of data belonging to each mode and then updating the model parameters.
Datasets
The paper evaluates the SMR approach using two types of datasets:
- Numerical Example: A synthetic dataset generated from a mixture of three Student's-t distributions, designed to mimic multimode behavior. Outliers (1%, 3%, and 5%) were intentionally added to test robustness.
- Real-life Industrial Dataset: Data from a primary reformer in an ammonia synthesis plant. This dataset captured oxygen concentration (the quality variable) and 13 process variables (secondary variables) over a period, exhibiting five distinct operating conditions (modes).
Key Insights and Ideas
- Student's-t is Key to Robustness: The paper strongly demonstrates that using the Student's-t distribution, rather than Gaussian, is critical for handling outliers. The heavier tails of the Student's-t distribution effectively "down-weight" the influence of outliers on parameter estimation.
- Joint Modeling of Input and Output: SMR's strength lies in modeling the joint probability distribution of input and output variables, which is more effective for multimode processes than methods that only cluster inputs.
- SMR Outperforms GMR in Noisy Conditions: Even when the number of components (modes) is correctly identified, SMR consistently shows better performance than GMR when outliers are present, especially as the outlier rate increases.
- Computational Efficiency is Acceptable: While SMR involves estimating an extra parameter (degrees of freedom) for the Student's-t distribution, which adds some computational cost, it remains within acceptable limits for practical industrial applications.
Key Results
- Numerical Example: SMR achieved significantly lower Root Mean Square Error (RMSE) on the testing dataset compared to Multi-DPLS and GMR, especially when outliers were present. For instance, with 3% outliers, SMR's RMSE was 1.5986, compared to 2.3787 for GMR and 4.2969 for Multi-DPLS.
- Primary Reformer Case Study: SMR again demonstrated superior performance. The determination coefficients for SMR were 0.9233, significantly higher than GMR (0.8655) and Multi-DPLS (0.7729) on the testing dataset. SMR also showed tighter scatters around the ideal prediction line in scatter plots.
- Robustness: The paper quantifies the deterioration of GMR's performance with increasing outliers, showing a 63.8% increase in RMSE from 1% to 5% outliers. SMR, in contrast, only saw a 7.7% increase in RMSE over the same range, highlighting its superior robustness.
- Computational Time: While SMR's training time (CPTtrn) was slightly higher than GMR and Multi-DPLS in the numerical example, the difference was negligible for online prediction (CPTtst). In the industrial case, SMR's training time was higher than GMR but still considered acceptable given its improved accuracy.
Limitations and Future Work
- Model Selection: The current SMR approach requires pre-defining the number of mixing components (K). Future work aims to develop methods for automatic model selection and parameter learning without needing a separate validation dataset.
- Time-Varying Processes: The paper acknowledges that performance might degrade in processes with time-varying characteristics.
- Adaptive SMR: The authors propose developing an adaptive Bayesian SMR (BSMR) that can handle time-variation by randomizing model parameters (including K) and updating them recursively online. This would allow the soft sensor to adapt to changing process conditions.
Summary updated