Sintering Quality Prediction Model Based on Semi-Supervised Dynamic Time Feature Extraction Framework

Yuxuan Li, Chunjie Yang, Youxian SunSensors 2022 · OpenReview

TL;DR

This paper introduces a novel semi-supervised framework (SS-DTFEE) to predict critical quality variables like FeO content in the iron ore sintering process. By leveraging abundant unlabeled data through a pre-training and fine-tuning strategy with an improved encoder-decoder architecture, the model significantly enhances prediction accuracy compared to traditional methods, addressing the challenge of scarce labeled industrial data.

What Problem Does This Paper Address?

The iron and steel industry, a cornerstone of the economy, faces significant challenges in optimizing its production processes, particularly in the energy-intensive sintering stage. A key issue is the difficulty in obtaining real-time measurements of crucial quality variables, such as the FeO content in sinter. This lack of timely information hinders effective process control, leading to unstable product quality and inefficient operations. Compounding this problem is the scarcity of labeled data, which are expensive and time-consuming to acquire through laboratory testing. Conventional soft sensor models, which aim to predict these hard-to-measure variables from readily available sensor data, struggle with this data limitation, resulting in poor performance. This paper tackles the challenge of building accurate soft sensors in industrial settings with limited labeled data by effectively utilizing abundant unlabeled process data.

Methods

The core of this work is the Semi-Supervised Dynamic Time Feature Extraction Framework (SS-DTFEE). This framework is built upon an encoder-decoder architecture that incorporates a novel weighted Bidirectional Long Short-Term Memory (BiLSTM) unit.

Here's a breakdown of the key methodological components:

  • Dynamic Time Feature Extraction (DTFEE): This initial step focuses on capturing the temporal dynamics inherent in industrial processes. It involves feature selection, time delay analysis, time difference calculation, and time serialization to create a comprehensive representation of the process data.
  • Weighted Bidirectional LSTM (wBiLSTM): Standard LSTMs can struggle with very long sequences. BiLSTMs address this by processing sequences in both forward and backward directions. The authors introduce a weighted BiLSTM, which allows for adjustable fusion of information from both directions, aiming for more flexible learning of temporal information.
  • Encoder-Decoder Architecture: This architecture is adapted from natural language processing for time-series prediction. An encoder processes the input sequence into a latent representation, and a decoder reconstructs the output sequence from this latent representation.
  • Semi-Supervised Learning (Pre-training and Fine-tuning): This is the crucial aspect for handling scarce labeled data.
  • Pre-training: The encoder-decoder model is first trained in an unsupervised manner on a large dataset of unlabeled process data. This phase allows the model to learn general patterns and hidden process information without relying on quality labels.
  • Fine-tuning: After pre-training, the learned model (or parts of it) is adapted using the limited labeled data. This involves migrating the learned parameters and then further training the model to predict the specific quality variable (e.g., FeO content). The authors explore strategies like freezing encoder weights and training the decoder to leverage the pre-trained knowledge efficiently.

Datasets

The study utilizes a real-world dataset collected from a sintering plant of an iron and steel enterprise.

  • Data Source: Collected from the Distributed Control System (DCS) of a 360 m² belt sintering machine over two months.
  • Data Characteristics: Approximately 101,400 records and 432 variables.
  • Labeled Data: A scarce set of 320 labeled data points for FeO content, obtained through manual sampling and testing (four times a day).
  • Unlabeled Data: The vast majority of the collected data, used for unsupervised pre-training.
  • Data Preprocessing: Outlier removal (3-σ criterion), 0-1 min-max normalization, and feature selection (using correlation coefficient and maximum mutual information coefficient) were applied. A total of 36 variables were selected as input features.

Key Insights and Ideas

  • Leveraging Unlabeled Data is Key: The scarcity of labeled industrial data is a significant bottleneck. This paper demonstrates that effectively utilizing abundant unlabeled data through unsupervised pre-training can dramatically improve model performance for tasks with limited labels.
  • Dynamic Temporal Features Matter: Industrial processes are inherently dynamic with time delays. The SS-DTFEE framework's emphasis on extracting these dynamic features (time delay, difference, serialization) is crucial for accurate prediction.
  • Weighted BiLSTM for Better Temporal Understanding: The proposed weighted BiLSTM offers a more nuanced way to capture bidirectional temporal dependencies compared to standard BiLSTMs, potentially leading to richer latent representations.
  • Pre-training and Fine-tuning Synergy: The combination of unsupervised pre-training to learn general process dynamics and supervised fine-tuning to adapt to specific quality prediction tasks is a powerful strategy for industrial soft sensing.
  • Encoder-Decoder as a Feature Extractor: The encoder-decoder structure, even when not fully fine-tuned, acts as an effective feature extractor, capturing complex process interactions that benefit downstream prediction.

Key Results

The proposed SS-DTFEE model achieved superior performance compared to baseline methods:

  • Hit Rate (HR@3%): The SS-DTFEE model achieved the highest Hit Rate of 0.813, significantly outperforming traditional LSTM (0.695) and BiLSTM (0.691) models. Models using only pre-training without fine-tuning also showed lower HR values (e.g., 0.739 for Encoder-decoder-LSTM+LSTM-PF).
  • Mean Square Error (MSE): The SS-DTFEE model achieved the lowest MSE of 0.048, indicating higher prediction accuracy.
  • Mean Absolute Error (MAE): The SS-DTFEE model achieved the lowest MAE of 0.182, further confirming its accuracy.
  • Qualitative Improvement: The paper shows that the fine-tuning strategy, especially with the BiLSTM encoder, leads to more stable and accurate predictions, as visualized in the prediction vs. actual plots.

Limitations and Future Work

  • Training Time: The proposed method involves training two models (pre-training and fine-tuning), which takes more time than training a single model.
  • Data Similarity: The fine-tuning model's performance is sensitive to the similarity between the pre-training and fine-tuning data. Significant changes in working conditions might affect the training effect.
  • Future Work: The authors suggest that the framework can be further improved in accuracy and applied to other process industries beyond sintering. They also highlight the potential for further research into optimizing the model architecture and training strategies.

Summary updated

Abstract

In the sintering process, it is difficult to obtain the key quality variables in real time, so there is lack of real-time information to guide the production process. Furthermore, these labeled data are too few, resulting in poor performance of conventional soft sensor models. Therefore, a novel semi-supervised dynamic feature extraction framework (SS-DTFEE) based on sequence pre-training and fine-tuning is proposed in this paper. Firstly, based on the DTFEE model, the time features of the sequences are extended and extracted. Secondly, a novel weighted bidirectional LSTM unit (BiLSTM) is designed to extract the latent variables of original sequence data. Based on improved BiLSTM, an encoder-decoder model is designed as a pre-training model with unsupervised learning to obtain the hidden information in the process. Next, through model migration and fine-tuning strategy, the prediction performance of labeled datasets is improved. The proposed method is applied in the actual sintering process to estimate the FeO content, which shows a significant improvement of the prediction accuracy, compared to traditional methods.
BibTeX
@inproceedings{li2022sintering,
  author = {Yuxuan Li and Chunjie Yang and Youxian Sun},
  title = {{Sintering Quality Prediction Model Based on Semi-Supervised Dynamic Time Feature Extraction Framework}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=VVVJpe3oKP},
}
Figure