Introduction
Predicting the state of smart energy systems, such as building HVAC or power grids, is challenging due to their complex, dynamic nature and sensitivity to external variables like weather. Traditional methods often rely on rigid mathematical models that are difficult to maintain as physical conditions evolve. This paper presents NEXTMon, a model-free monitoring architecture that leverages "nexting"—a biologically inspired reinforcement learning approach—to predict future system states across multiple timescales simultaneously.
System Architecture and Methodology
The NEXTMon system operates as an advisory tool within a human-in-the-loop framework. Rather than controlling the physical system directly, it provides real-time predictions to a human operator, who then makes informed control decisions.
The core of NEXTMon is the application of Multi-timescale Nexting. This method uses Temporal Difference (TD) learning to predict future sensor values by treating raw signals as "pseudo-rewards." To handle continuous, high-dimensional data, the system employs tile coding, which discretizes the state space into overlapping grids to create efficient, sparse binary feature vectors. By adjusting the discount rate ($\gamma$), the algorithm can simultaneously maintain weight vectors for different future horizons (e.g., short-term vs. long-term trends).
Evaluation and Results
The system was evaluated using simulations of a water tank and a thermal house model. The house model incorporated 20 days of real-world hourly temperature data from Berlin to simulate realistic environmental fluctuations.
Key findings include:
- Learning Efficiency: The system required approximately 100 hours of operation to populate its weight vectors and achieve stable, accurate predictions.
- Predictive Accuracy: Once trained, NEXTMon's predictions closely tracked ideal offline benchmarks, successfully anticipating temperature trends despite significant outdoor fluctuations.
- Anticipatory Monitoring: The system demonstrated the ability to predict the actions of existing controllers, such as identifying heater "switch-off" events before they occurred.
Conclusion and Future Work
NEXTMon offers a flexible, model-free alternative for energy system monitoring. By learning the "physics" of a building through direct observation, it eliminates the need for complex custom engineering. While the initial "blind" period and high-dimensional scaling remain challenges, the system's data efficiency and real-time adaptability make it a promising tool for complex domains like distributed wind power generation. Future work will focus on improving user interfaces for multi-timescale forecasts.
Summary updated