On emulating hardware/software co-designed control algorithms for packet switches

Dimitris Syrivelis, Paolo Giaccone, Iordanis Koutsopoulos, Marco Pretti, Leandros Tassiulas

Abstract

Testing complex network control algorithms requires balancing software flexibility with hardware performance. This paper introduces a hybrid emulation framework that bridges this gap by combining the NetFPGA platform with the Click Modular Router. By accounting for real-world constraints like bus latency and FPGA resource limits, the framework provides a realistic environment for evaluating co-designed hardware/software schedulers. A case study using a Belief Propagation (BP) multicast scheduler demonstrates that hardware acceleration can significantly outperform software-only solutions, even when accounting for significant communication overhead.

Introduction

Modern high-speed packet switches, operating at 10 Gbps or higher, face extremely tight decision windows—often less than 51 nanoseconds—to schedule incoming packets. While simple heuristics can meet these deadlines, they often sacrifice throughput, particularly for complex tasks like multicast scheduling, which is NP-hard. Researchers typically face a choice between flexible but slow software simulations and rigid, expensive hardware implementations. This paper presents an emulation framework designed to evaluate co-designed algorithms that leverage both CPU flexibility and FPGA speed.

The Emulation Architecture

The proposed framework consists of four primary units designed to simulate a realistic network environment:

  1. Hardware Emulator Unit (NetFPGA): An FPGA-based board hosting the hardware-accelerated control logic.
  2. Software Datapath (Click Modular Router): A flexible software framework on the host CPU managing packet forwarding and queues.
  3. Traffic Generator Toolchain: An offline utility that prepares high-speed traffic patterns for injection.
  4. Performance Measurement Module: A dedicated system for tracking throughput and latency.
High-level architecture of the hybrid hardware/software emulation framework showing the interaction between hardware components and the Linux kernel.

The architecture explicitly models the interaction between the host CPU and the hardware accelerator via the PCI bus, allowing researchers to measure the "hidden" costs of data movement that are often ignored in pure software simulations.

Hardware/Software Interface Trade-offs

A significant finding of this work is that the interface (PCI bus/DMA) often represents a larger bottleneck than the algorithm itself. The authors explored two primary methods for data transfer: Packet-based I/O and Register-based I/O.

Comparison of Register-based and Packet-based I/O interfaces within the FPGA logic, highlighting the resource-intensive nature of the packet segmentation engine.

While Packet-based I/O (utilizing DMA) is approximately 20 times faster, it consumes significantly more FPGA resources. Due to the complexity of the multicast scheduler logic, the authors were forced to use Register-based I/O to fit the design onto the legacy NetFPGA 1G hardware. This reduced the communication subsystem's footprint from 6,265 slices to just 2,580 slices, freeing up the necessary space for the control logic.

Case Study: DEC-BPn Multicast Scheduler

To validate the framework, the authors implemented a Belief Propagation (BP) algorithm for multicast scheduling. BP is an iterative message-passing algorithm typically used in AI. The authors developed a "decimated" version (DEC-BPn) optimized for hardware state machines. By unrolling loops and utilizing parallel logic, the hardware implementation achieved a 3x speedup over a high-end Intel i7 CPU.

Key Results and Insights

The emulation revealed critical performance metrics for the co-designed system:

  • Execution Speed: The hardware version of the DEC-BP2 scheduler completed its task in 7.06 μs, compared to roughly 21 μs for the software version on a 3.06 GHz Intel i7.
  • System Latency: For a 4x4 switch, total timeslot duration (including data transfer and execution) ranged from 40.77 μs to 44.12 μs.
  • Communication Overhead: Pushing data to the hardware took approximately 23 μs, while the actual complex BP calculation took only 4–7 μs, confirming the interface as the primary bottleneck.

Conclusion

The presented emulation framework provides a practical middle ground for network researchers. It successfully demonstrates that hardware/software co-design can solve complex scheduling problems at speeds unattainable by software alone, while also exposing the critical role of interface latency and hardware resource management in system design. Future work will focus on scaling these algorithms to larger switch fabrics using more modern FPGA platforms like NetFPGA 10G to alleviate current PCI bottlenecks.

Summary updated

BibTeX
@inproceedings{syrivelis2014emulating,
  author = {Dimitris Syrivelis and Paolo Giaccone and Iordanis Koutsopoulos and Marco Pretti and Leandros Tassiulas},
  title = {{On emulating hardware/software co-designed control algorithms for packet switches}},
  booktitle = {SimuTools},
  year = {2014},
  doi = {10.4108/icst.simutools.2014.254797},
}
Figure