IoTreeplay: Synchronous Distributed Traffic Replay in IoT Environments

Markus Toll, Ilja Behnke, Odej Kao

TL;DR

Testing complex Internet of Things (IoT) systems requires replaying realistic network traffic across multiple devices simultaneously. This paper introduces IoTreeplay, a tool that extends existing traffic replay capabilities to enable synchronous, distributed TCP traffic replay across an entire IoT network, allowing for more comprehensive and application-independent system testing.

What Problem Does This Paper Address?

The Internet of Things (IoT) is rapidly expanding, with numerous interconnected, heterogeneous devices forming complex networks. For critical applications, thoroughly testing these systems before deployment is paramount to ensure safety, efficiency, and scalability. While existing tools can emulate network properties, they often fall short in providing fine-grained, network-wide traffic control necessary to test applications independently of their environment. Specifically, replaying realistic, synchronized network traffic across multiple IoT devices simultaneously, mimicking real-world conditions, is a significant challenge. This is particularly true for distributed IoT networks where a lack of central control makes it difficult to predict system behavior under various traffic loads or network changes.

Methods

IoTreeplay builds upon the existing TCPLivePlay tool, enhancing its capabilities for distributed IoT environments. The core idea is to allow the synchronous replaying of captured TCP traffic across multiple endpoints within a user-defined network or testbed.

Here's a breakdown of the key methods:

  • Distributed Replay: IoTreeplay orchestrates multiple TCPLivePlay instances running on different nodes in the network. A central controller manages the process, splitting the original traffic capture into individual connections.
  • Synchronized Timing: The tool ensures that traffic is replayed synchronously across all nodes. This is achieved by calculating and applying offsets based on original packet timestamps and synchronizing the start times of replay instances.
  • TCP Traffic Emulation: IoTreeplay focuses on replaying TCP traffic. It injects packets at the network interface level, simulating TCP connections without establishing actual ones. This allows for the replay of various application-level protocols that use TCP (e.g., MQTT, Modbus).
  • IP and Port Rewriting: To adapt the captured traffic to the replay environment, IP addresses and ports are rewritten. Checksums are recalculated accordingly.
  • Error Correction and Modifications: The authors identified and fixed several issues in the TCPLivePlay version they used, including a faulty timeout logic that prevented packet processing and incorrect sequence/acknowledgment number calculations. They also implemented delay management by using usleep() based on packet timestamps to achieve more accurate replay timing.
  • Controller-Node Architecture: The system comprises a controller responsible for input processing, file splitting, distribution, and recording retrieval, and multiple "nodes" (e.g., Raspberry Pis) that execute the replay and capture traffic.

Datasets

The paper does not explicitly mention using pre-existing public datasets for training or evaluation. Instead, the authors generated their own traffic traces for testing. They mention using "multiple test runs" that included "two-endpoint-connections with an transmission rate as well as multi-endpoint-connections with up to two hundred connections from publicly available repositories." This suggests they either captured traffic from real-world IoT scenarios or synthesized representative traffic patterns.

Key Insights and Ideas

  • Bridging the Gap in IoT Testing: IoTreeplay effectively addresses the need for application-independent network testing in distributed IoT environments, a gap identified in existing frameworks.
  • Synchronicity is Crucial: The ability to replay traffic synchronously across multiple devices is key to accurately simulating real-world distributed system behavior, especially for time-sensitive IoT applications.
  • Outliers Drive Deviations: The evaluation revealed that a small number of connections are responsible for the majority of replay deviations. This suggests that understanding and mitigating these outlier scenarios is critical for improving overall replay accuracy.
  • Connection Length Matters: Shorter connections tend to exhibit higher replay accuracy. This could be because they require less time to stabilize and are less susceptible to cumulative timing errors.
  • Limitations of Standard TCP Replay: The tool's struggle with deviations from regular TCP connections (like packet loss or resets) highlights the inherent challenges in perfectly replaying stateful protocols in dynamic environments.

Key Results

The evaluation focused on the accuracy of replay in terms of packet timestamps.

  • Good Overall Accuracy: The results show good accuracy for most connections, with average and median deviations generally within 0-60 ms.
  • Impact of Outliers: Outliers, representing significant deviations, were observed, but the authors attribute most of these to technical limitations of the test setup and network, rather than fundamental flaws in IoTreeplay itself.
  • Performance Limitations: The replay was not capable of keeping up with very high packet rates (over 100 packets/second) in some scenarios, indicating a performance limitation of the current implementation rather than a general failure.
  • Scalability: The authors suggest that the distributed nature of IoTreeplay, where replay is split across nodes, indicates good scalability potential, with performance primarily limited by individual node processing power and network specifications.

Limitations and Future Work

  • TCP Focus: IoTreeplay currently only supports TCP traffic replay. The authors suggest expanding support to other protocols like UDP as a significant future direction.
  • Error Handling: The tool struggles with replaying traffic that already contains duplicate packets or packet loss from the original connection, sometimes leading to aborted connections. Mitigating this by filtering such packets or improving the tool's ability to distinguish original anomalies from replay-induced ones is a potential area for improvement.
  • Timestamp Precision: While usleep() was used for delay management, the authors note that the precision is ultimately bound by the capabilities of pcap.h and unistd.h libraries.
  • Future Work: The authors propose several avenues for future work:
  • Integrating IoTreeplay with network emulators to test system resilience under various network conditions (delay, packet loss, etc.).
  • Exploring scalability questions by testing how existing devices handle increased network loads.
  • Utilizing IoTreeplay for security applications, such as injecting traffic to test overload or vulnerability scenarios.
  • Expanding protocol support beyond TCP.

Summary updated

Abstract

Use-cases in the Internet of Things (IoT) typically involve a high number of interconnected, heterogeneous devices. Due to the criticality of many IoT scenarios, systems and applications need to be tested thoroughly before rollout. Existing staging environments and testing frameworks are able to emulate network properties but fail to deliver actual network-wide traffic control to test systems application independently. To extend existing frameworks, we present the distributed traffic replaying tool IoTreeplay. The tool embeds TCPLivePlay into an environment that allows the synchronous replaying of network traffic with multiple endpoints and connections. Replaying takes place in a user-defined network or testbed containing IoT use-cases. Network traffic can be captured and compared to the original trace to evaluate accuracy and reliability. The resulting implementation is able to accurately replay connections within a maximum transmission rate but struggles with deviations from regular TCP connections, like packet loss or connection reset. An evaluation has been performed, measuring individual and aggregated delays between packets, based on the recorded timestamps.
BibTeX
@inproceedings{toll2022iotreeplay,
  author = {Markus Toll and Ilja Behnke and Odej Kao},
  title = {{IoTreeplay: Synchronous Distributed Traffic Replay in IoT Environments}},
  booktitle = {IC2E},
  year = {2022},
  url = {https://openreview.net/forum?id=0C8cO8kNJi},
}
Figure