Abstract
Prior research has produced numerous systems aimed at securing smart home environments. However, the practical efficacy of these systems depends on their ability to be tested against realistic scenarios that reflect actual human behavior. This paper presents Helion, a system that leverages n-gram language modeling to learn the regularities of user-defined smart home routines. By treating home automation as a structured language, Helion predicts "natural" event sequences that mimic real-world usage. We further introduce HelionHA, an integration with the Home Assistant platform that enables the end-to-end execution of these generated scenarios as test cases for real and virtual smart home devices.
Introduction
As smart home ecosystems grow in complexity, ensuring their safety and security has become a critical challenge. Security researchers typically evaluate smart home defense systems using one of two methods: random event permutations or real execution traces. Both approaches are fundamentally flawed. Random sequences (e.g., unlocking a door immediately followed by opening the blinds and turning on an oven) rarely reflect human logic, leading to impractical security evaluations. Conversely, collecting real execution traces is highly privacy-invasive and often contains "noise"—device-specific technical glitches that obscure the semantic intent of the user.
Helion addresses this bottleneck by generating synthetically realistic scenarios. By focusing on the "routines" (IF-THEN rules) users program into their homes, Helion captures the underlying intent of home automation while preserving privacy and filtering out technical noise.
The Helion Framework
Helion is built on the "naturalness of code" hypothesis, which suggests that just as human-written software follows predictable patterns, so do the automation routines created for smart homes. The framework utilizes a four-step pipeline to transform abstract user intent into executable test cases.
1. Data Collection and the HOME Corpus
To train its models, the authors developed the HOME Corpus. They surveyed 40 participants to collect 273 routines (233 unique) expressed in plain English. These routines were then tokenized into a standardized format representing device types, attributes, and states.
2. Informed Scheduling
Static routines do not inherently contain the chronological data needed for testing. Helion employs a novel "Informed Scheduling" step, using execution indicators (metadata regarding how often and when rules typically run) to expand these routines into a month-long sequence of events. For the HOME Corpus, this resulted in a dataset of over 30,000 events.
3. N-gram Language Modeling
Helion uses statistical n-gram models ($n \ge 3$) to learn the regularities within these event sequences. By assuming the Markov property—where the probability of an event depends on a fixed window of preceding events—the model learns to predict the next logical step in a home automation sequence.
4. Scenario Prediction: "Up" and "Down" Flavors
Helion can generate two distinct types of scenarios based on the model's probability distributions:
- "Up" Flavor (Natural): The model predicts the most probable next events. These are used for "positive testing" to ensure security systems do not interfere with normal, everyday home usage.
- "Down" Flavor (Unrealistic): The model predicts the least probable (but still syntactically valid) events. This is a powerful tool for "negative testing," allowing researchers to discover safety violations that occur during rare, unexpected "perfect storm" conditions.
HelionHA: From Theory to Execution
To bridge the gap between statistical modeling and physical testing, the authors developed HelionHA. This extension for the open-source Home Assistant platform allows researchers to execute Helion’s predicted scenarios on actual hardware or virtualized device instances.
A key component of this integration is the parse_token.py script, which maps abstract model tokens (e.g., <light, switch, ON>) to specific entity IDs within a Home Assistant configuration. This allows for a real-time, end-to-end exploration of scenarios where researchers can monitor device states on a dashboard as the Helion-generated test case unfolds.
Key Insights
- The Importance of Context: The authors found that models with $n < 3$ (unigrams or bigrams) fail to capture the semantic logic of a home. For example, a sequence like "Motion Detected $\rightarrow$ Lights On $\rightarrow$ Door Unlocked" requires at least a 3-gram to maintain the necessary context for a realistic workflow.
- Routines as Semantic Source Code: By training on user-defined routines rather than raw sensor data, Helion focuses on the intent of the smart home. This approach effectively bypasses the noise of hardware-level events and focuses on the high-level logic where security vulnerabilities often reside.
Conclusion and Future Work
Helion provides a scalable, privacy-preserving method for generating realistic smart home test data. While the current implementation uses n-gram models for their computational efficiency, future iterations may explore Transformer-based architectures to capture even longer-range dependencies in user behavior. Additionally, expanding the HOME Corpus will be essential to capturing the vast and evolving vocabulary of modern smart home devices. By providing both the dataset and the HelionHA platform to the community, the authors have established a new foundation for the rigorous testing of smart home security systems.
Summary updated