Abstract
Data drift evaluation is crucial in the operational step in the industry. In the real world, several drift types are usually contributing to drift detection, which may come from input data and output data distributions. In addition, the application context and the interpretation of these drift types add complexity to drift analysis. In this work, we apply drift detection in the specific domain of electrical transmission network systems. Three drift types, covariate, label, and concept drift, are considered and implemented on systems based on Physics-Informed Neural Networks (PINNs). The experimental results show the impact of each drift type and the evolution of their contributions when drift occurs in the industrial system. A contextual interpretation of the obtained results is also developed in this specific application domain for the three drift types.
1. Introduction
In real-world industrial deployments, AI models are rarely static. They operate in environments that evolve over time, a phenomenon known as data drift. When a model's performance begins to decay in production, engineers face the challenge of identifying the root cause. Is the input data fundamentally different (Covariate Shift)? Has the distribution of the target variable changed (Label Shift)? Or has the underlying relationship between inputs and outputs broken down (Concept Drift)?
This problem is particularly acute in safety-critical infrastructure like electrical transmission networks. A failure in a model predicting power flow—perhaps due to an unseen change in grid topology—can lead to significant economic losses or even cascading blackouts. This research aims to bridge the gap between simple drift detection and a nuanced understanding of distributional shifts in AI-hybrid models that incorporate physical laws.
2. Methodology
The researchers implemented a three-way drift decomposition framework based on the joint distribution $P(X, Y) = P(Y|X)P(X)$.
- Covariate Shift ($P(X)$ changes): Detected using Maximum Mean Discrepancy (MMD) for continuous features (e.g., power production) and Chi-Squared tests for categorical features (e.g., line connectivity).
- Label Shift ($P(Y)$ changes): Monitored via the Kolmogorov-Smirnov test to detect changes in target power flow distributions.
- Concept Drift ($P(Y|X)$ changes): Identified through residual analysis against a baseline, tracking shifts in the underlying "physics" learned by the model.
The study compared three architectures: a standard Multi-Layer Perceptron (MLP), an MLP with physics-based regularization, and a Graph Neural Network (GNN)-based PINN that explicitly encodes grid topology.
3. Experimental Results and Insights
Experiments were conducted using power flow simulators across various grid scales. A key "stress test" involved the Under-Drift-Distribution (UDD) dataset, which featured simultaneous line disconnections never seen during training.
Topology as Covariate Shift
The results demonstrate that in power grids, changing line connectivity is mathematically equivalent to a massive covariate shift. In the UDD scenario, covariate shift accounted for 95.7% of the total drift. The GNN-based PINN showed remarkable resilience, achieving a Mean Absolute Error (MAE) of 2e-2, compared to 4e-1 for the standard MLP—a 20x difference in accuracy under drift.
Physics Preservation
Beyond accuracy, the PINN models maintained higher "physics compliance." While standard MLPs began predicting power flows that violated basic conservation laws under drift, the GNN-based approach remained closer to physical reality because its architecture is inherently invariant to topological changes.
4. Conclusion
Decomposing drift into covariate, label, and concept components provides a clearer diagnostic for model decay in industrial systems. The findings highlight that PINNs are significantly more robust to the topological shifts common in power grids. Future work will focus on integrating dimensionality reduction techniques like PCA or VAEs to capture more complex, non-linear drift patterns and developing real-time monitoring tools for grid operators.
Summary updated