Enhancing Human-AI Collaboration through a Conversational Agent for Energy Efficiency

Marco Brambilla, Riccardo Campi, Mathyas Giudici, Nicolò Oreste Pinciroli Vago, Marco Brambilla 0001, Piero FraternaliAAAI 2025 · DOI

Abstract

Large Language Models (LLMs) offer significant potential for interactive energy efficiency (EE) advice but are often limited by hallucinations and a lack of provenance. This paper introduces ENERGENIUS Guru, a conversational agent that leverages a Knowledge Graph-based Retrieval-Augmented Generation (RAG) architecture to provide grounded, localized advice. By adhering to Human-Centered AI (HCAI) principles, the system aims to improve user trust and self-efficacy in navigating complex energy regulations. Evaluation by domain experts demonstrates high accuracy and successful adaptation to regional contexts.

1. Introduction

The transition to sustainable energy consumption requires empowering residential users with accurate, actionable information. While LLMs can facilitate fluid dialogue, their application in specialized domains like Energy Efficiency (EE) is hindered by two primary issues: hallucinations (confidently stating false information) and a lack of provenance (inability to cite sources). Inaccurate advice regarding energy regulations or financial incentives can lead to costly errors or user frustration. Furthermore, EE policies are highly localized, requiring systems to distinguish between different national regulatory frameworks (e.g., Italy vs. Switzerland). This research addresses these challenges by developing a Human-Centered AI (HCAI) collaborator that prioritizes technical accuracy and user empowerment.

2. Methodology

The proposed system, ENERGENIUS Guru, employs a multi-layered architecture designed to move beyond "black box" AI models.

2.1 Multi-layered Architecture

The system decouples the conversational "brain" from the underlying data and decision-making logic. As shown in Figure 1, the agent interacts with three primary external components to ensure context-aware responses:

  • User Context: Stores profiles, geographic location, and local regulations.
  • Business Logic: Contains rules for decision-making and coordination.
  • Knowledge Base: A structured repository of domain knowledge, definitions, and relationships.
Conceptual architecture of the ENERGENIUS Guru system, illustrating the interaction between the agent, users, and external modules.

2.2 KG-based RAG and HCAI Design

Unlike standard RAG systems that retrieve information from unstructured text chunks, ENERGENIUS Guru utilizes a Knowledge Graph (KG). The agent uses an LLM to translate user queries into SPARQL queries, retrieving structured facts from official documents. This approach ensures that responses are grounded in verified data and allows for "language agnosticism," where the underlying facts remain consistent across different languages.

The UI/UX design is grounded in Shneiderman’s HCAI framework, focusing on providing clear feedback and maintaining user control. The interface includes history management and clear indicators of the conversation's progress to increase the user's sense of "self-efficacy."

Wireframe of the conversational interface designed to support user trust and self-efficacy through clear history and status indicators.

3. Evaluation and Results

The system was evaluated using a curated dataset of 101 question-answer pairs derived from official energy documents in Italy and Switzerland. Four energy-domain experts assessed the agent using the RAGAs framework, focusing on faithfulness, relevance, and context relevance.

Key results include:

  • Overall Accuracy: The agent provided correct and relevant content in 75.2% ± 2.7% of cases.
  • Context Adaptation: The system demonstrated robust performance across languages, with 77.4% accuracy for Italian queries and 73.0% for English.
  • Regulatory Precision: The agent successfully handled country-specific legal questions for Italy (73.4%) and Switzerland (71.2%).

4. Discussion and Key Insights

The evaluation highlighted that user trust is significantly bolstered by traceability. Providing citations for advice allows users to verify information, which is critical in high-stakes domains like energy management. Furthermore, the "mixed-initiative" approach positions the agent as a tool to augment human capability, helping users navigate complex regulations that would otherwise be too tedious to research manually.

5. Limitations and Future Work

While promising, the current study is limited by a small evaluation scale and a reliance on expert feedback rather than broad user studies. Some expert feedback suggested that the advice provided was occasionally too general. Future work will focus on integrating real-time IoT data, such as smart meter readings, to provide more personalized and actionable recommendations. Additionally, the knowledge base will be expanded to include more countries to test the system's scalability.

Summary updated

BibTeX
@inproceedings{brambilla2025enhancing,
  author = {Marco Brambilla and Riccardo Campi and Mathyas Giudici and Nicolò Oreste Pinciroli Vago and Marco Brambilla and Piero Fraternali},
  title = {{Enhancing Human-AI Collaboration through a Conversational Agent for Energy Efficiency}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaaiss.v5i1.35554},
}
Figure