Ensemble Learning Model for Mechanical Performance Prediction of Strip and Its Reliability Evaluation

板带力学性能预报的集成学习模型及 其可靠性评价, Li Fei-Fei, Yong Song, Chao Liu, Bo LiJournal of Mechanical Engineering 2021 · DOI

TL;DR

This paper introduces an ensemble learning approach using BP neural networks to predict the mechanical properties of hot-rolled strips with improved accuracy and, crucially, a novel reliability evaluation metric. By shifting from direct property prediction to predicting property deviations between samples and analyzing the dispersion of these predictions, the model offers a more robust and trustworthy tool for industrial applications.

What Problem Does This Paper Address?

Predicting the mechanical properties (like yield strength, tensile strength, and elongation) of hot-rolled strips is vital for manufacturing quality control and product development. However, existing prediction models often suffer from low accuracy, poor generalization to new data, and a critical lack of reliability assessment. This means that even if a model provides a prediction, there's no way to know how confident that prediction is, limiting its practical use in dynamic industrial environments where data quality can vary. The authors aim to bridge this gap by developing a model that not only predicts properties accurately but also quantifies the certainty of its predictions.

Methods

The core of the proposed method involves two key innovations:

  1. Deviation Prediction Modeling: Instead of directly predicting a mechanical property value, the authors transform the problem into predicting the difference or deviation in mechanical properties between pairs of samples. This is achieved using a Backpropagation (BP) neural network. The input to the BP network consists of the feature vectors of two samples, and the output is their mechanical property deviation, encoded in a binary format. This approach is hypothesized to better capture the underlying relationships and variations in the data.
  2. Ensemble Learning with Reliability Evaluation:
    • Sub-model Creation: Multiple BP neural networks (sub-models) are trained on different randomly sampled subsets of the training data. This is a standard practice in ensemble learning to improve generalization.
  3. Bagging: The Bagging algorithm is employed to combine these sub-models. Bagging involves training each sub-model on a bootstrap sample (sampling with replacement) of the training data.
  4. Reliability Metric (PICP): A novel reliability evaluation index, Prediction Interval Coverage Probability (PICP), is introduced. For a given test sample, the ensemble of sub-models generates a set of predictions. The mean of these predictions is taken as the final predicted value. The dispersion (variance or spread) of these predictions is then used to calculate the PICP. PICP quantifies the proportion of predictions that fall within a defined error interval around the mean. A higher PICP indicates greater confidence in the prediction.

Datasets

The study utilizes production data from a Compact Strip Production (CSP) hot-rolling line. The dataset includes 31 features related to chemical composition and key process parameters. For modeling, Principal Component Analysis (PCA) was used for dimensionality reduction, resulting in 13 principal components and 3300 data points for training. The models were evaluated on a separate test set of 200 samples, with the entire process repeated five times to ensure robustness.

Key Insights and Ideas

  • Shifting Focus to Deviations: Predicting property deviations rather than absolute values appears to be a more effective way to model the complex relationships in hot-rolling data, leading to better generalization.
  • PICP as a Trustworthiness Indicator: The PICP metric provides a quantifiable measure of prediction reliability. It allows users to distinguish between predictions with high certainty and those with significant uncertainty, which is crucial for practical decision-making.
  • Identifying "Unreliable" Samples: The PICP metric can effectively identify a small percentage of samples for which the model cannot provide a reliable prediction. This allows for targeted manual inspection or alternative quality control measures for these specific cases.
  • Ensemble Learning Synergy: Combining multiple deviation-predicting BP networks through Bagging significantly enhances both prediction accuracy and generalization, outperforming individual sub-models.

Key Results

  • The ensemble learning model achieved high prediction accuracy:
  • Yield strength and tensile strength within ±30 MPa error: 98.45% and 98.97% accuracy, respectively.
  • Elongation within ±5% error: 99.48% accuracy.
  • The PICP metric proved effective. When PICP exceeded 45% for yield strength, predictions were within ±30 MPa with high confidence. For the ensemble model, a PICP of 55% could narrow the prediction error to within ±20 MPa for yield strength, with an average error of 6.62 MPa.
  • The model successfully identified approximately 3.5% of samples with higher uncertainty, while the remaining samples were predicted with high accuracy.

Limitations and Future Work

  • Identification of Uncertain Samples: While the model identifies uncertain predictions, these "unreliable" samples still require manual intervention or further inspection in production settings. The authors suggest that optimizing the PICP threshold can balance prediction accuracy with the number of samples requiring manual checks.
  • Generalizability to Other Steel Grades: The current study used data from a specific CSP hot-rolling line without differentiating steel grades. Future work could explore the model's performance and adaptability across different steel types.
  • Further Optimization of Ensemble Strategy: While Bagging was used, other ensemble methods could be explored to potentially further improve performance.
  • Real-time Implementation: The paper focuses on model development and validation. Future work could involve integrating this model into real-time production systems for immediate feedback and control.

Summary updated

Abstract

摘要: 热轧板带力学性能预报具有十分广阔的应用前景。但目前的力学性能预报模型精度不高、泛化能力较差,且无法评价预测结果的准确性,限制了其在实际应用中的效果。为了提高力学性能预报模型的精度和实现预测结果的可靠性评价,采用BP神经网络将目前直接预测力学性能结果的建模方法转换为对样本间的力学性能偏差分布预测的建模,并结合模型预测结果分布的离散程度设计可靠性评价指标,同时采用集成学习提高模型的泛化能力。通过试验验证,该集成学习模型具有较高的预测精度,进一步,根据可靠性评价指标分析,除了其中占比约3.5%样本的预测结果具有较大不确定性,剩余样本的屈服强度和抗拉强度在误差±30 MPa的预测准确率达到了98.45%和98.97%,延伸率在误差±5%下的预测准确率达到了99.48%,有效地提高了模型预测准确率,在生产现场应用中具有一定的指导意义。
BibTeX
@article{2021ensemble,
  author = {板带力学性能预报的集成学习模型及 其可靠性评价 and Li Fei-Fei and Yong Song and Chao Liu and Bo Li},
  title = {{Ensemble Learning Model for Mechanical Performance Prediction of Strip and Its Reliability Evaluation}},
  journal = {Journal of Mechanical Engineering},
  year = {2021},
  doi = {10.3901/jme.2021.02.239},
}
Figure