Abstract
Smart systems are characterised by their ability to analyse measured data in live and to react to changes according to expert rules. Therefore, such systems exploit appropriate data models together with actions, triggered by domain-related conditions. The challenge at hand is that smart systems usually need to process thousands of updates to detect which rules need to be triggered, often even on restricted hardware like a Raspberry Pi. Despite various approaches have been investigated to efficiently check conditions on data models, they either assume to fit into main memory or rely on high latency persistence storage systems that severely damage the reactivity of smart systems. To tackle this challenge, we propose a novel composition process, which weaves executable rules into a data model with lazy loading abilities. We quantitatively show, on a smart building case study, that our approach can handle, at low latency, big sets of rules on top of large-scale data models on restricted hardware.
Introduction
Smart systems, such as those managing smart buildings, must process a continuous stream of sensor data to trigger reactive actions based on predefined business rules. Traditionally, this reactivity is managed by Complex Event Processing (CEP) or rule engines like Drools. However, these solutions often encounter a "memory wall," as they typically require the entire data model and rule set to reside in main memory for performance. On embedded hardware with limited RAM, such as a Raspberry Pi, this requirement severely limits system scalability. Conversely, relying on traditional persistent databases introduces latency that compromises real-time reactivity. This paper presents a method to bridge this gap by "weaving" executable rules directly into the system's data model using a [email protected] approach, enabling high-throughput processing with a minimal memory footprint.
Methodology: Rule Weaving and Lazy Loading
The proposed approach integrates rules into the Kevoree Modeling Framework (KMF) through a novel composition process. This methodology rests on three primary pillars:
- Rule Weaving: Instead of maintaining rules in a separate engine, they are woven into the model's API. By overriding the "setter" methods of the data model, any update to a value (e.g., a sensor reading) automatically triggers the relevant rule checks within the model itself.
- Lazy Loading: Leveraging KMF’s capabilities, the system loads only the necessary fragments of a model from persistent storage (such as RocksDB) into memory on demand. This applies to both the data elements and the rule structures, which are represented as Abstract Syntax Trees (ASTs).
- Two-Step Compilation: The process involves generating an "Actions Dictionary" where rule actions are compiled into static Java methods, followed by generating trigger code that links specific model attributes to rule conditions.
The rule language is inspired by Gremlin, allowing for efficient graph-based navigation and manipulation. By representing rules as ASTs within the model graph, rule processing is treated as a standard, highly optimized graph navigation task.
Evaluation and Results
The approach was evaluated using a smart building dataset with model sizes scaled from 100,000 to 5,000,000 elements to test the limits of the lazy loading mechanism on restricted hardware.
Memory Efficiency
A significant finding of this research is the achievement of a near-constant memory footprint. Because of the lazy loading mechanism, the memory required is determined by the size of the rule currently being processed rather than the total size of the model or rule set. In tests involving 5 million model elements, memory consumption remained stable and generally below 50MB after initial loading phases.
Throughput Performance
Despite the inherent overhead of loading data from persistent storage, the system maintained high performance. Evaluation showed a throughput of 41,000 to 70,000 rules per second, depending on the complexity of the rule conditions. This demonstrates that the engine can comfortably run on a Raspberry Pi, maintaining real-time performance while managing a massive rule-base.
Conclusion and Future Work
By embedding rules directly into the data model's setters, the overhead of a separate reasoning engine is eliminated, making the data structure itself the mechanism for reactivity. This allows for large-scale smart systems to operate on resource-constrained devices. Future work will focus on expanding the rule language to support complex multi-attribute conditions, implementing the Rete algorithm for further optimization, and introducing temporal logic to enable reactions based on historical data patterns.
Summary updated