Abstract
Large Language Models (LLMs) offer significant potential for interactive energy efficiency (EE) advice but are often limited by hallucinations and a lack of provenance. This paper introduces ENERGENIUS Guru, a conversational agent that leverages a Knowledge Graph-based Retrieval-Augmented Generation (RAG) architecture to provide grounded, localized advice. By adhering to Human-Centered AI (HCAI) principles, the system aims to improve user trust and self-efficacy in navigating complex energy regulations. Evaluation by domain experts demonstrates high accuracy and successful adaptation to regional contexts.
1. Introduction
The transition to sustainable energy consumption requires empowering residential users with accurate, actionable information. While LLMs can facilitate fluid dialogue, their application in specialized domains like Energy Efficiency (EE) is hindered by two primary issues: hallucinations (confidently stating false information) and a lack of provenance (inability to cite sources). Inaccurate advice regarding energy regulations or financial incentives can lead to costly errors or user frustration. Furthermore, EE policies are highly localized, requiring systems to distinguish between different national regulatory frameworks (e.g., Italy vs. Switzerland). This research addresses these challenges by developing a Human-Centered AI (HCAI) collaborator that prioritizes technical accuracy and user empowerment.
2. Methodology
The proposed system, ENERGENIUS Guru, employs a multi-layered architecture designed to move beyond "black box" AI models.
2.1 Multi-layered Architecture
The system decouples the conversational "brain" from the underlying data and decision-making logic. As shown in Figure 1, the agent interacts with three primary external components to ensure context-aware responses:
- User Context: Stores profiles, geographic location, and local regulations.
- Business Logic: Contains rules for decision-making and coordination.
- Knowledge Base: A structured repository of domain knowledge, definitions, and relationships.
2.2 KG-based RAG and HCAI Design
Unlike standard RAG systems that retrieve information from unstructured text chunks, ENERGENIUS Guru utilizes a Knowledge Graph (KG). The agent uses an LLM to translate user queries into SPARQL queries, retrieving structured facts from official documents. This approach ensures that responses are grounded in verified data and allows for "language agnosticism," where the underlying facts remain consistent across different languages.
The UI/UX design is grounded in Shneiderman’s HCAI framework, focusing on providing clear feedback and maintaining user control. The interface includes history management and clear indicators of the conversation's progress to increase the user's sense of "self-efficacy."
3. Evaluation and Results
The system was evaluated using a curated dataset of 101 question-answer pairs derived from official energy documents in Italy and Switzerland. Four energy-domain experts assessed the agent using the RAGAs framework, focusing on faithfulness, relevance, and context relevance.
Key results include:
- Overall Accuracy: The agent provided correct and relevant content in 75.2% ± 2.7% of cases.
- Context Adaptation: The system demonstrated robust performance across languages, with 77.4% accuracy for Italian queries and 73.0% for English.
- Regulatory Precision: The agent successfully handled country-specific legal questions for Italy (73.4%) and Switzerland (71.2%).
4. Discussion and Key Insights
The evaluation highlighted that user trust is significantly bolstered by traceability. Providing citations for advice allows users to verify information, which is critical in high-stakes domains like energy management. Furthermore, the "mixed-initiative" approach positions the agent as a tool to augment human capability, helping users navigate complex regulations that would otherwise be too tedious to research manually.
5. Limitations and Future Work
While promising, the current study is limited by a small evaluation scale and a reliance on expert feedback rather than broad user studies. Some expert feedback suggested that the advice provided was occasionally too general. Future work will focus on integrating real-time IoT data, such as smart meter readings, to provide more personalized and actionable recommendations. Additionally, the knowledge base will be expanded to include more countries to test the system's scalability.
Summary updated