Abstract
Streaming Data-Driven Optimization (SDDO) is essential for managing real-time systems like traffic flow and energy grids. However, these systems are frequently disrupted by "concept drift," where underlying data distributions shift unpredictably. Traditional drift detectors, largely optimized for classification tasks, struggle with the complex, real-valued landscapes of optimization problems and often fail to generalize across different tasks. This paper presents TRACE (TRAnsferable Concept-drift Estimator), a novel framework that redefines drift detection as a sequence modeling task. By processing statistical features of surrogate model errors through a dual-attention encoder, TRACE learns to recognize universal drift patterns. Our evaluation shows that TRACE significantly outperforms traditional methods in precision and robustness, enabling more reliable optimization in dynamic environments.
Introduction: The Challenge of Concept Drift in SDDO
In Streaming Data-Driven Optimization (SDDO), algorithms must continuously adapt to a stream of incoming data. A primary obstacle is concept drift—sudden or gradual changes in the data distribution that render previous optimization strategies obsolete. Without effective detection, optimizers risk overfitting to outdated information, leading to severe performance degradation.
Existing drift detection methods typically rely on hand-crafted statistical rules designed for discrete classification labels. These methods often fail in SDDO because optimization landscapes are unbounded and continuous. Furthermore, traditional detectors lack transferability; a detector tuned for one dataset rarely performs well on another without significant reconfiguration.
The TRACE Framework
TRACE addresses these limitations by treating drift detection as a learnable sequence modeling problem. The architecture is composed of three integrated stages:
- Stream Tokenization: Rather than monitoring raw data, TRACE tracks the prediction errors of a surrogate model. It uses a sliding window to extract statistical features (mean, variance, quartiles), converting the error stream into a sequence of informative "tokens."
- Dual-Attention Encoder: This core component utilizes two specialized mechanisms:
- Global Multi-Head Self-Attention (G-MSA): Identifies long-range structural dependencies across the sequence.
- Context-Guided MSA (C-MSA): Compares recent data against a "context token" representing the stable environment, allowing the model to distinguish between noise and actual distributional shifts.
- Drift Classification Head: A pointer-like network that determines if a drift has occurred and identifies its precise location in the sequence.
To demonstrate its utility, the authors also introduced TRACE-EA, an evolutionary algorithm that uses TRACE as a plug-and-play module to trigger knowledge transfer and model updates immediately upon drift detection.
Experimental Evaluation and Results
TRACE was evaluated against standard benchmarks (SDDObench, DBG, and GMPB) and real-world datasets (e.g., Electricity, Kddcup99).
Detection Performance
TRACE consistently outperformed established detectors such as ADWIN and KSWIN. It achieved higher precision and F1 scores, demonstrating a unique ability to minimize false alarms while maintaining high sensitivity to actual shifts. By training on diverse synthetic streams, TRACE acquired a generalized understanding of drift that transferred effectively to "Out-of-Distribution" (OOD) optimization tasks.
Optimization Robustness
When integrated into evolutionary algorithms, TRACE-EA significantly reduced the Dynamic Tracking Error ($E_{DT}$). It detected drifts faster than competing methods, allowing the optimizer to reset or adapt before performance plummeted. In real-world stream clustering tasks, TRACE-EA maintained superior clustering quality (lower DBI scores) and exhibited lower performance variance, proving its readiness for practical deployment.
Conclusion
TRACE represents a significant shift in drift detection for streaming optimization. By moving from fixed statistical tests to a learnable, attention-based architecture, it provides a transferable solution that adapts to diverse and unseen optimization landscapes. Future work will focus on reducing detection latency through adaptive windowing and exploring tighter integration between the detector and the optimizer's core logic.
Summary updated