A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case

Yinghao Zhu, Chenhao Ying, Yan Zhou, Jiangang LuIEEE Access 2024 · DOI

Introduction

Predicting outcomes in complex industrial processes, such as rubber mixing, presents a significant challenge for conventional machine learning models. These environments generate multivariable time series data characterized by high levels of noise and "covariate shift"—where the statistical properties of data change over time due to machinery wear or raw material variability. Standard models, including Transformers, often struggle with these "messy" real-world scenarios because they fail to capture local dependencies and are sensitive to irrelevant features.

To address these issues, this paper introduces AdaDynaTrans, a novel attention-based model designed for robust industrial time series prediction. By integrating dynamic feature selection, position-enhanced attention, and a "worst-case" segmental training strategy, AdaDynaTrans achieves state-of-the-art accuracy in predicting Mooney viscosity, a critical metric in tire production.

Methodology: The AdaDynaTrans Architecture

The AdaDynaTrans model is composed of several specialized modules designed to handle the unique complexities of industrial data.

Dynamic Feature Selection

The model utilizes Dynamic Residual Units (DRU) and Feature Focused Units (FFU). These components act as intelligent filters, employing gating mechanisms to automatically suppress irrelevant input features. This ensures that the model focuses on the most predictive variables while adjusting its depth based on the complexity of the incoming data.

Position Enhanced Transformer (PET)

A core innovation is the Position Enhanced Transformer (PET). Standard Transformers rely on absolute position encodings, which can lead to "disorderly" attention patterns in time series. PET replaces this with a directed graph-based approach, treating time steps as nodes. This allows the model to explicitly learn the importance of the relative distance between any two points in time, significantly improving its ability to capture temporal dependencies.

Boosting-based Importance Compute in the PET module.

Temporal Segmentation and Segmental Training (TSST)

To combat covariate shift, the authors developed the Temporal Segmentation and Segmental Training (TSST) strategy. This method uses a "maximum entropy" principle to divide training data into diverse segments with distinct distributions. By training the model to find "common knowledge" that performs well across these varied segments, it becomes more resilient to shifts in unseen data.

The TSST strategy for maximizing distributional diversity across training segments.

Static Covariate Extraction

Unlike models that treat static metadata (e.g., equipment IDs) as simple inputs, the Static Covariate Extraction Module (SCEM) extracts context vectors. These vectors are used to initialize the hidden states of temporal modules, providing a stable anchor for the model's predictions.

Experimental Evaluation

The model was rigorously tested on two datasets:

  1. Mooney Viscosity Dataset: An industrial dataset comprising 100,000 samples from a tire factory, featuring six temporal variables (temperature, power, etc.) and five static covariates.
  2. Air Quality Dataset: A public benchmark containing hourly data from Beijing to evaluate the model's generalizability.

Key Results

AdaDynaTrans consistently outperformed established baselines, including GRU, LSTM, LightGBM, and AdaRNN.

  • Industrial Performance: On the Mooney viscosity task, the model reduced Mean Absolute Error (MAE) by 51% and Root Mean Square Error (RMSE) by 49.5% compared to a standard GRU baseline.
  • Public Benchmark: It achieved an average improvement of 49.13% on the air quality dataset.
  • Ablation Study: Removing the Temporal Variable Evolution Module (TVEM) resulted in a 27.66% spike in RMSE, highlighting the importance of modeling feature evolution over time.

Conclusion and Future Directions

AdaDynaTrans demonstrates that combining relative position awareness with adversarial-style segmental training can significantly improve the reliability of industrial time series forecasting. While the model excels in accuracy, future work will focus on enhancing interpretability for factory operators and extending the architecture to handle spatiotemporal data from distributed sensor networks.

Summary updated

BibTeX
@article{zhu2024predictive,
  author = {Yinghao Zhu and Chenhao Ying and Yan Zhou and Jiangang Lu},
  title = {{A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case}},
  journal = {IEEE Access},
  year = {2024},
  doi = {10.1109/access.2024.3365499},
}
Figure