Data-Driven Learning-Based Optimization for Distribution System State Estimation

Ahmed S. Zamzam, Xiao Fu, Nicholas D. SidiropoulosIEEE Transactions on Power Systems 2019 · DOI · arXiv

Abstract

Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss-Newton perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravates the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss-Newton is also known to be sensitive to initialization. This paper proposes using historical or simulation-derived data to train a shallow neural network to “learn to initialize,” mapping available measurements to a point in the neighborhood of the true latent states (network voltages) to initialize Gauss-Newton. This hybrid machine learning/optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency compared to conventional optimization-only approaches.

The Challenge of Distribution System State Estimation

State Estimation (SE) acts as the "GPS" of the power grid, allowing operators to monitor real-time voltages and power flows. While well-established for high-voltage transmission lines, it is notoriously difficult for distribution systems due to three primary factors:

  1. Measurement Scarcity: Distribution grids have far fewer real-time sensors than transmission grids.
  2. Non-Convexity: The physics of power flow leads to a non-convex optimization problem with multiple local minima.
  3. Initialization Sensitivity: The industry-standard solver, Gauss-Newton, is extremely sensitive to its starting point. A poor initial guess can lead to convergence at the wrong solution or total divergence.

Proposed Methodology: Learning to Initialize

The authors propose a two-stage hybrid framework that combines the global search capabilities of machine learning with the local precision of classical optimization.

Overview of the proposed hybrid DSSE framework, showing the training phase and the estimation phase where the neural network initializes the state estimator.

Stage 1: The Neural Network Initializer

A shallow neural network with a single hidden layer is trained to map available measurements (from PMUs, smart meters, and pseudo-measurements) to an approximate system state. In the test case using the IEEE-37 distribution feeder, the network processed 103 measurements to estimate 210 state variables (voltage phasors).

Stage 2: Gauss-Newton Refinement

The output of the neural network is used as the initial guess for a standard Gauss-Newton solver. This stage "polishes" the estimate to ensure it strictly satisfies the physical laws of the network.

The "Lazy" ε-Insensitive Cost Function

A key innovation is the use of an ε-insensitive cost function for training: $$\min \sum \max(\|v^j - g_T(z^j)\|_2^2 - \epsilon^2, 0)$$ This approach only requires the network to get the estimate into the right "neighborhood" (a ball of radius ε) of the true state.

Visualization of the ε-insensitive cost function for different values of ε, illustrating how it ignores small errors to improve generalization.

By lowering the accuracy requirement for the neural network, the authors reduce the complexity of the model and improve its ability to generalize. Theoretical analysis (Proposition 1) shows that allowing this small error exponentially reduces the number of neurons needed to represent the mapping.

Key Results

The hybrid approach was validated against a state-of-the-art complex-variable Gauss-Newton (G-N) solver on the IEEE-37 benchmark:

  • Stability: In 1,000 test cases, the standard G-N solver diverged 28 times, while the proposed hybrid method achieved zero divergences.
  • Accuracy: The estimation error was nearly 10 times lower for the proposed method ($9.5 \times 10^{-3}$ vs $9.8 \times 10^{-2}$).
  • Speed: The hybrid approach was roughly 7x faster (347ms vs 2468ms) because the superior initialization allowed the Gauss-Newton stage to converge in far fewer iterations.

Conclusion

The "learning to initialize" paradigm offers a robust solution to the non-convexity and data scarcity inherent in distribution systems. While major structural grid changes might require retraining, the model's ε-insensitive objective makes it surprisingly robust to modest topology reconfigurations. Future research will explore applying this hybrid concept to other non-convex power system problems, such as Optimal Power Flow (OPF).

Summary updated

Abstract

Distribution system state estimation (DSSE) is a core task for monitoring and control of distribution networks. Widely used algorithms such as Gauss-Newton perform poorly with the limited number of measurements typically available for DSSE, often require many iterations to obtain reasonable results, and sometimes fail to converge. DSSE is a non-convex problem, and working with a limited number of measurements further aggravates the situation, as indeterminacy induces multiple global (in addition to local) minima. Gauss-Newton is also known to be sensitive to initialization. Hence, the situation is far from ideal. It is therefore natural to ask if there is a smart way of initializing Gauss-Newton that will avoid these DSSE-specific pitfalls. This paper proposes using historical or simulation-derived data to train a shallow neural network to “learn to initialize,” that is, map the available measurements to a point in the neighborhood of the true latent states (network voltages), which is used to initialize Gauss-Newton. It is shown that this hybrid machine learning/optimization approach yields superior performance in terms of stability, accuracy, and runtime efficiency, compared to conventional optimization-only approaches. It is also shown that judicious design of the neural network training cost function helps to improve the overall DSSE performance.
BibTeX
@article{zamzam2019data,
  author = {Ahmed S. Zamzam and Xiao Fu and Nicholas D. Sidiropoulos},
  title = {{Data-Driven Learning-Based Optimization for Distribution System State Estimation}},
  journal = {IEEE Transactions on Power Systems},
  year = {2019},
  doi = {10.1109/tpwrs.2019.2909150},
}
Figure