OMG: A Scalable and Flexible Simulation and Testing Environment Toolbox for Intelligent Microgrid Control

Stefan Heid, Daniel Weber, Henrik Bode, Eyke Hüllermeier, Oliver Wallscheid, Daniel Weber 0004J. Open Source Softw 2020 · DOI · DBLP

The transition toward renewable energy systems has accelerated the deployment of micro- and smart-grids (MSGs). These grids are essential for integrating volatile power sources and providing electricity to remote areas. However, controlling MSGs is a complex challenge. They are safety-critical, highly variable in topology, and rely on power electronic converters that operate on extremely fast time scales.

Existing open-source tools like PyPSA or Pandapower are excellent for transmission network planning but typically focus on steady-state behavior. They often miss "transients"—the split-second fluctuations that can lead to equipment failure or instability. For researchers applying reinforcement learning (RL), there has been a lack of a sandbox that combines high-fidelity electrical physics with the ease of use found in modern AI frameworks.

The OMG Framework Architecture

To address this gap, the authors developed the OpenModelica Microgrid Gym (OMG). OMG acts as a translation layer between the high-fidelity physics of electrical circuits and the standardized interfaces used in modern machine learning.

Overview of the OMG toolbox workflow, showing the integration between the OpenModelica simulation environment and the OpenAI Gym RL agent interface.

The toolbox utilizes a two-pronged approach:

  1. The Physics Backend (OpenModelica): OMG uses OpenModelica to handle the heavy lifting of solving the ordinary differential equations (ODEs) that describe electrical circuits. This allows for component-oriented modeling with high-fidelity dynamics.
  2. The AI Interface (OpenAI Gym): Using the Functional Mock-up Interface (FMI 2.0) standard, the OpenModelica models are wrapped into a Python-compatible format. By implementing the OpenAI Gym interface, OMG allows users to plug in industry-standard RL libraries like Stable Baselines3 or TF-Agents without rewriting underlying simulation code.

High-Fidelity Simulation for Intelligent Control

A key insight of the OMG toolbox is its focus on transient-level fidelity. While many RL environments for power systems simplify physics to accelerate training, OMG preserves component-level dynamics. This is crucial for verifying that an AI controller will function safely in real-world conditions where transients are unavoidable.

Furthermore, the toolbox supports hierarchical control. It includes building blocks for classical inner and primary control levels, enabling researchers to compare "intelligent" AI agents against "gold standard" classical controllers (such as droop or PID control) on a level playing field. This emphasis on safety-first learning acknowledges that in power systems, the "move fast and break things" philosophy of traditional AI development is not viable.

Versatility and Benchmarking

As a generative environment, OMG provides more than just static datasets. It offers predefined example grids that serve as benchmarks, parameterizable components for custom sources and loads, and support for multi-phase AC and DC operation.

The toolbox also includes built-in monitoring tools to track the "live" performance of an RL agent, mapping grid behavior as the agent's parameters evolve during training. Because it adheres to the FMI standard, models can be easily exchanged between different computing platforms, enhancing interoperability.

Conclusion and Future Directions

While OMG currently faces some limitations, such as computational overhead due to its high-fidelity nature and limited native OS support for Mac, it represents a significant step forward. It provides a foundation for creating standardized benchmarks where different RL algorithms can be compared under identical, rigorous physical conditions. Future work will likely focus on optimizing simulation speed and expanding the library of predefined microgrid scenarios to further support the development of robust, intelligent grid controllers.

Summary updated

BibTeX
@article{heid2020omg,
  author = {Stefan Heid and Daniel Weber and Henrik Bode and Eyke Hüllermeier and Oliver Wallscheid and Daniel Weber},
  title = {{OMG: A Scalable and Flexible Simulation and Testing Environment Toolbox for Intelligent Microgrid Control}},
  journal = {J. Open Source Softw.},
  year = {2020},
  doi = {10.21105/joss.02435},
}
Figure