TL;DR
OpenMeter is a privacy-preserving multi-agent system (MAS) that allows non-technical grid operators to query complex smart meter databases using natural language. By orchestrating open-weight models locally, the system achieves significant accuracy gains on resource-constrained hardware while ensuring sensitive consumer energy data never leaves the utility's infrastructure.
What Problem Does This Paper Address?
Modern power grids are drowning in data. Smart meters generate a continuous stream of telemetry—voltage, active energy, reactive power—at 10–15 minute intervals. While this data is vital for anomaly detection and grid optimization, it’s often trapped behind complex SQL databases. Grid operators are domain experts, not database engineers; this creates a "sensory overload" where actionable insights are inaccessible without technical intervention.
Furthermore, consumer energy data is highly sensitive. Adversaries can use it to infer daily routines or even individual identities with up to 99% accuracy. This privacy risk prohibits the use of popular cloud-based LLMs (like GPT-4). The challenge, therefore, is to build a system that is both highly accurate at "Text-to-SQL" translation and capable of running entirely on local, private hardware.
Methods
The authors introduce OpenMeter, a schema-agnostic Multi-Agent System designed with a "Plan-then-Execute" pattern. The architecture (shown in Figure 1) splits the workload across three specialized agents:
- Orchestrator (O): Acts as the central coordinator. It classifies the user's intent, creates an execution plan, and performs "LLM-as-Curator"—dynamically filtering a massive domain knowledge base to provide only the relevant context to the next agent.
- Coding Agent (CA): Translates the plan into executable DuckDB SQL or Python code. Crucially, it features an autonomous self-correction loop; if the code fails, it uses database error logs to fix itself without bothering the Orchestrator.
- Interpretation Agent (IA): Takes the raw data results and translates them back into human-readable, domain-grounded explanations (e.g., "This voltage drop suggests a transformer overload").
This structural separation mitigates the "Reasoning-Action Mismatch" often seen in monolithic LLM setups, where a model might lose track of the logic while trying to write code.
Datasets
The researchers evaluated OpenMeter using:
- Industrial Smart Meter Data: A real-world dataset from an industrial partner containing ~7.5 million rows across three different database schemas (EMSify, LP3, and LPR).
- Evaluation Golden Dataset: A newly introduced benchmark of 62 manually verified Natural Language-to-SQL pairs derived from actual operational workflows. This dataset covers 13 analytical categories and three difficulty levels.
Key Insights and Ideas
- Orchestration as a Force Multiplier for Small Models: The biggest "aha moment" was that full multi-agent orchestration is most beneficial on smaller, local models (like Qwen 3.5 9B). On resource-constrained hardware, the MAS approach provided a +6.1 percentage point boost in accuracy over the best single-agent setup.
- LLM-as-Curator: Instead of using a standard RAG (Retrieval-Augmented Generation) pipeline with vector databases, the Orchestrator simply "curates" the knowledge base during its inference pass. This prevents "attention degradation"—where the model gets confused by too much irrelevant info—without adding the latency of external retrieval.
- The "Human Gap" in Evaluation: The study found that automated metrics (like "Exact Match" of SQL code) are far too conservative. Human evaluators accepted alternative SQL formulations at much higher rates than automated judges, suggesting that current benchmarks systematically underestimate the practical utility of these systems.
Key Results
The authors tested the system across two tiers: Tier 1 (Local 9B model) and Tier 2 (Cloud 397B model).
- Accuracy: In the local Tier 1 environment, the full multi-agent setup (M+Full) achieved the highest execution accuracy (42.4% Relaxed Execution Rate).
- Self-Correction: The retry loop was a game-changer for smaller models, recovering 22.4 percentage points in executability (ESR) in Tier 1. For the massive Tier 2 model, which rarely made syntax errors, the gains were marginal.
- Latency: There is a "coordination tax." The multi-agent setup is significantly slower (183s end-to-end) compared to a single-agent baseline (22s), primarily due to the multiple LLM calls and the injection of domain knowledge.
Limitations and Future Work
While OpenMeter proves the viability of local, private grid analytics, it has limitations:
- Single-Operator Focus: The evaluation was limited to data from one industrial partner.
- Zero-Shot Only: The agents were not provided with "few-shot" examples (previous successful queries), which usually boosts LLM performance.
- Latency: The 3-minute response time for complex queries might be too slow for real-time emergency response, though it is a massive improvement over the "hours to days" currently required for manual database engineering.
Future work will focus on expanding the corpus across multiple utility operators and fine-tuning the agents specifically on energy-domain NL-to-SQL pairs to further reduce latency and improve accuracy.
Summary updated