Network Technologies Used to Aggregate Environmental Data

Paul Stasiuk, Konstantin Läufer, George K. ThiruvathukalLoyola 2013

Abstract: This Presentation is brought to you for free and open access by the Faculty Publications at Loyola eCommons. It has been accepted for inclusion in Computer Science: Faculty Publications and Other Works by an authorized administrator of Loyola eCommons.

Introduction

As the Internet of Things (IoT) expands, the primary challenge shifts from simple data collection to efficient aggregation from numerous "edge" devices to a central system. Traditional environmental monitoring often relies on monolithic systems or standard web protocols that may struggle to scale under high-frequency updates.

The Loyola Weather Service (lws) project addresses this by proposing a scalable architecture for environmental monitoring and autonomous control. By benchmarking communication protocols, the research demonstrates that specialized messaging via ZeroMQ offers more than double the performance of standard RESTful HTTP for high-volume sensor data aggregation.

System Architecture

The researchers developed a three-tier architecture designed for scalability and flexibility:

  1. Nodes: These are the edge devices responsible for data collection and near-real-time response.
  2. Aggregation Layer: A scalable Software-as-a-Service (SaaS) layer that collects data from all nodes, stores it in a MongoDB database, and performs analysis.
  3. Frontend Layer: The user-facing interface and API that allows interaction with the aggregated data.
The three-tier architecture of the Loyola Weather Service (lws), consisting of Frontend, Aggregation, and Node layers.

By separating the frontend from the aggregation layer, the system can scale horizontally. If the API becomes a bottleneck, it can be migrated to dedicated hardware without disrupting the data collection flow.

Hardware and Software Implementation

The "Nodes" layer is comprised of Raspberry Pi hosts paired with Phidget I/O kits for sensor interfacing. These nodes capture environmental metrics such as temperature and humidity in a JSON format.

Schematic of the Raspberry Pi hardware used as the core of the lws sensor nodes.

The software stack is built primarily on Python, utilizing Flask for the RESTful API and ZeroMQ for high-performance asynchronous messaging. A document-based database, MongoDB, was selected for the aggregation layer to handle diverse sensor types without the constraints of a rigid SQL schema.

Performance Evaluation

The core of the research involved benchmarking two communication protocols: REST (Representational State Transfer) and ZeroMQ. While REST is the industry standard for web interoperability, it carries significant overhead for high-frequency sensor updates.

For evaluation, synthetic benchmarking loads ranging from 100 to 100,000 requests were used to measure latency and throughput.

Key Results

  • Throughput: ZeroMQ consistently outperformed REST. At a scale of 100 requests, ZeroMQ was approximately 4x faster (2.06s vs. 8.38s). At 100,000 requests, ZeroMQ completed the task in ~3,541 seconds, compared to ~4,987 seconds for REST.
  • Protocol Efficiency: The authors concluded that ZeroMQ provides at least double the performance over HTTP-based protocols for large volumes of node queries due to its priority message queuing.
  • Database Indexing: Benchmarks for database range queries were less conclusive. Using compound time indices versus standard ISO time indices showed only marginal performance increases.

Conclusion and Future Work

The research highlights that protocol selection is critical for IoT scalability. While standard web protocols are convenient, specialized messaging layers like ZeroMQ are essential for high-volume data pipelines.

Future work aims to transition the lws from a monitoring system to an active control framework. The goal is to support autonomous environmental management, such as adjusting lighting and HVAC systems based on real-time sensor data. The open-source nature of the project encourages community contributions to extend functionality to new sensor and actuator types.

This summary is based on research by Paul Stasiuk, Konstantin Läufer, and George K. Thiruvathukal from Loyola University Chicago.

Summary updated

Abstract

This Presentation is brought to you for free and open access by the Faculty Publications at Loyola eCommons. It has been accepted for inclusion in Computer Science: Faculty Publications and Other Works by an authorized administrator of Loyola eCommons. For more information, please contact
BibTeX
@inproceedings{stasiuk2013network,
  author = {Paul Stasiuk and Konstantin Läufer and George K. Thiruvathukal},
  title = {{Network Technologies Used to Aggregate Environmental Data}},
  booktitle = {Loyola},
  year = {2013},
}
Figure