Communication channel prioritization in a publish-subscribe architecture

Ali Paikan, Daniele Domenichelli, Lorenzo Natale

Abstract

Real-time communication is vital in distributed applications where timing constraints on data processing and task execution are fundamental. Current standards-based software engineering does not yet specify how real-time properties should be integrated into publish-subscribe middleware. This article describes an approach for integrating priority Quality of Service (QoS) into a publish-subscribe middleware by leveraging operating system functionalities to prioritize specific communication channels at runtime. The approach is implemented in the YARP (Yet Another Robot Platform) framework, and preliminary performance results are presented.

Introduction

In complex distributed systems, such as humanoid robots, software components are typically decoupled using a publish-subscribe model. While this architecture enhances modularity and scalability, it often lacks the fine-grained control necessary for real-time message delivery. A common challenge arises when critical control signals (e.g., a "stop" command) must share network resources with high-bandwidth, low-priority traffic (e.g., video streams). Without prioritization, critical messages can experience significant latency and jitter, which is unacceptable for robotic control loops. This paper proposes a "channel prioritization" approach that integrates Quality of Service (QoS) controls directly into the middleware by leveraging existing operating system and network primitives.

Methodology

The researchers implemented their approach within the YARP (Yet Another Robot Platform) middleware. The strategy operates at two distinct levels to ensure end-to-end prioritization:

  1. Thread Prioritization: At the OS level, YARP was modified to allow developers to assign specific priorities to threads handling individual communication channels. On Linux, this utilizes the SCHED_FIFO real-time scheduling policy, ensuring that high-priority "write" operations are processed by the CPU before lower-priority tasks.
  2. Packet Prioritization: At the network level, the system utilizes Type of Service (TOS) or Differentiated Services Code Point (DSCP) bits in the IP header. These bits signal to network drivers and QoS-enabled hardware switches which packets should be prioritized in transmission queues.
Thread and packet prioritization model in YARP

These properties are configurable dynamically at runtime via administrative commands, allowing the system to adapt without requiring a restart. This "thin" middleware approach avoids the overhead of a centralized broker while providing robust prioritization.

Experimental Results

The approach was evaluated using a physical testbed consisting of three Linux nodes with PREEMPT-RT kernels connected via a Cisco Catalyst 2960 QoS-enabled switch. The experiments compared "Normal" traffic against "QoS-enabled" traffic under various network loads.

Average packet trip time at 20% load, comparing normal and QoS-enabled configurations

As shown in the figure above, even at a relatively low load of 20%, the QoS-enabled channels ("qos" bars) demonstrate lower average trip times compared to standard channels ("normal" bars) across both UDP and TCP protocols. The benefits were even more pronounced at 70% load, where non-prioritized TCP traffic saw latencies spike to nearly 150ms, while prioritized channels maintained significantly lower and more deterministic trip times. This reduction in jitter is crucial for maintaining stable control loops in distributed robotic applications.

Conclusion

The proposed channel prioritization scheme demonstrates that effective QoS can be achieved in publish-subscribe middleware by correctly mapping application-level priorities to existing OS and network features. While this "best-effort" QoS does not provide the absolute guarantees of hard real-time fieldbuses, it significantly improves the performance and determinism of standard Ethernet for high-level robotic tasks. Future work will explore the use of autonomous administrative agents to dynamically manage these priorities based on the robot's operational state.

Summary updated

Abstract

Real-Time communication are important in all those distributed applications where timing constraints on data proccessing and task executation play a fundamental role. Standards-base software engineering does not yet specify how real-time properties should be integrated into a publish/subscribe middleware. This article describes an approach for integration of priority quality of service in a publish/subscribe middleware. The approach simply leverages the operating system functionalities to provide a framework where specific communication channels can be prioritized at run-time. The quality of service is implemented in YARP (Yet Another Robot Platform) framework and the primarily results of performance tests are presented.
BibTeX
@inproceedings{paikan2015communication,
  author = {Ali Paikan and Daniele Domenichelli and Lorenzo Natale},
  title = {{Communication channel prioritization in a publish-subscribe architecture}},
  booktitle = {SEARIS},
  year = {2015},
  url = {https://openreview.net/forum?id=qpERplsZTn},
}
Figure