AutoML for multi-class anomaly compensation of sensor drift

Melanie Schaller, Mathis Kruse, Antonio Ortega, Marius Lindauer, Bodo Rosenhahn

Introduction

In industrial environments, sensors are subject to degradation over time due to aging, environmental fluctuations, or chemical poisoning—a phenomenon known as sensor drift. This drift causes the underlying data distribution to shift, leading to a significant decline in the predictive accuracy of machine learning models. A critical observation made by the authors is that standard evaluation techniques, such as 10-fold cross-validation, are fundamentally unsuitable for this problem. By randomly shuffling data, these techniques allow models to "cheat" by training on future drift patterns, resulting in over-optimistic performance metrics that do not hold up in real-world deployments.

The Problem: Sensor Drift and the Cross-Validation Trap

Sensor drift is not merely additive noise; it is a complex, non-linear transformation of the feature space. As shown in Figure 2, a decision boundary optimized for data at time $t_1$ may fail to correctly classify data at time $t_2$ because the clusters have shifted, leading to "incorrectly predicted" regions.

Visualization of sensor drift over time causing decision boundary shifts.

The authors demonstrate that a standard Random Forest model achieving a 98% F1-score under 10-fold cross-validation plummets to just 56% when evaluated on a realistic chronological split. This "cross-validation trap" masks the vulnerability of models to temporal shifts, highlighting the need for more rigorous training and evaluation paradigms.

Proposed Solution: Novel Training Paradigms

To bridge the gap between laboratory evaluation and industrial reality, the authors propose a two-pronged approach. First, they introduce a novel training paradigm that enforces strict chronological data splitting.

Comparison of traditional cross-validation vs. novel chronological training paradigms and the AutoML framework.

As illustrated in Figure 1, the "Old training setup" (cross-validation) is replaced by two realistic alternatives:

  1. AD Training Setup: The model is trained on early, "normal" data and tested on later, "drifted" data.
  2. Incremental Training Setup: This setup simulates real-time data streaming, where the model is updated sequentially as new batches of data (from $t=1$ to $t=n$) become available.

AutoML-DC: Automated Drift Compensation

The second part of the solution is AutoML-DC (Drift Compensation). Rather than manually engineering a single model to account for drift, the authors utilize an Automated Machine Learning framework based on auto-sklearn. This framework explores a massive search space to find optimal combinations of data preprocessing, algorithm selection, and ensemble configurations.

Key components of AutoML-DC include:

  • Automated Preprocessing: Techniques such as imputation, feature scaling, and polynomial feature generation are automatically selected. Ablation studies showed that preprocessing is the most critical factor, contributing significantly to the model's ability to untangle drift from the signal.
  • Diverse Ensembles: The system naturally converges on ensembles (often dominated by Random Forests) with varying regularization strengths. This diversity allows the model to capture both short-term fluctuations and long-term drift trends.
  • Meta-learning: By leveraging knowledge from previous datasets, the framework can jump-start its hyperparameter search, significantly reducing computational overhead.

Experimental Results and Discussion

The framework was evaluated using the Vergara et al. Gas Sensor Drift Dataset, which contains 36 months of data from a 16-element sensor array.

Under the strict chronological evaluation (training on batches 1-5, testing on 6-10), AutoML-DC achieved an F1-score of 76%, outperforming traditional baselines like kNN and Random Forest (56%) and specialized ensemble methods (29%). The stability of the approach was also notable, with a standard deviation of less than 0.3% across multiple runs.

Ablation analysis further revealed that removing automated preprocessing dropped the F1-score to 61%, while removing meta-learning dropped it to 63%. These results underscore that the "Auto" in AutoML is essential for handling the complexities of physical sensor drift.

Conclusion

This research highlights a major pitfall in current sensor drift literature and provides a robust framework for addressing it. By moving away from random cross-validation toward chronological splitting and leveraging AutoML for implicit drift compensation, practitioners can build models that are far more reliable in long-term industrial deployments. Future work will focus on expanding these methods to unsupervised settings where ground-truth labels for drifted data are unavailable.

Summary updated

Abstract

Addressing sensor drift is essential in industrial measurement systems, where precise data output is necessary for maintaining accuracy and reliability in monitoring processes, as it progressively degrades the performance of machine learning models over time. Our findings indicate that the standard cross-validation method used in existing model training overestimates performance by inadequately accounting for drift. This is primarily because typical cross-validation techniques allow data instances to appear in both training and testing sets, thereby distorting the accuracy of the predictive evaluation. As a result, these models are unable to precisely predict future drift effects, compromising their ability to generalize and adapt to evolving data conditions. This paper presents two solutions: (1) a novel sensor drift compensation learning paradigm for validating models, and (2) automated machine learning (AutoML) techniques to enhance classification performance and compensate sensor drift. By employing strategies such as data balancing, meta-learning, automated ensemble learning, hyperparameter optimization, feature selection, and boosting, our AutoML-DC (Drift Compensation) model significantly improves classification performance against sensor drift. AutoML-DC further adapts effectively to varying drift severities. • Our analysis demonstrates that the conventional training configurations are suboptimal in learning and compensating for sensor drift. Thus, we propose a novel sensor drift compensation learning training paradigm that closely matches real-world scenarios. • Our findings further indicate, that AutoML techniques along with the proposed training paradigm enable effective drift adaptation to evolving levels of drift severity and complex drift dynamics in sensor measurements. • By utilizing meta-learning, AutoML-DC starts from initial configurations based on prior data, lowering the extensive data requirements normally needed for training neural networks or ensemble models. • We make use of AutoML techniques to enhance model robustness (see standard deviation) by combining multiple models to capture diverse data patterns, optimizing feature selection, and preventing overfitting through smart training termination. • We conduct extensive benchmarking experiments against existing models and highlight the significant accuracy improvements realized when adopting AutoML-DC in practical drift compensation scenarios in industrial measurements.
BibTeX
@inproceedings{schaller2025automl,
  author = {Melanie Schaller and Mathis Kruse and Antonio Ortega and Marius Lindauer and Bodo Rosenhahn},
  title = {{AutoML for multi-class anomaly compensation of sensor drift}},
  booktitle = {Measurement},
  year = {2025},
  doi = {10.1016/j.measurement.2025.117097},
}
Figure