Ultra-Fast Downloading of Partial Bitstreams through Ethernet

Pierre Bomel, Jérémie Crenne, Linfeng Ye, Jean-Philippe Diguet, Guy GogniatARCS 2009 · DOI

TL;DR

This paper presents a novel, ultra-fast partial bitstream downloading method for FPGAs over standard Ethernet. By introducing a specific data-link layer protocol and optimizing hardware/software partitioning, they achieve download speeds up to 20 times faster than existing solutions, while requiring minimal on-chip memory.

What Problem Does This Paper Address?

Field-Programmable Gate Arrays (FPGAs) are increasingly used in diverse applications, from embedded systems to spacecrafts. Their ability to be partially reconfigured (PR) at runtime allows for dynamic adaptation of functionality, reducing component count, size, weight, and power consumption. However, a significant bottleneck for PR is the time it takes to download the partial bitstreams (the configuration data) into the FPGA. Current methods often rely on local storage (FLASH, RAM) or slow network protocols, leading to latencies that can negate the benefits of PR, especially in time-critical applications. This paper addresses the challenge of significantly accelerating this bitstream download process over standard Ethernet networks, making dynamic reconfiguration more practical and efficient.

Methods

The core contribution of this paper lies in a new data-link layer protocol designed for ultra-fast partial bitstream downloading over Ethernet. This protocol, combined with a carefully optimized hardware and software architecture, aims to maximize the data transfer rate.

Key aspects of their approach include:

  • Custom Data-Link Layer Protocol: Instead of relying on standard network protocols like TCP/IP, which introduce significant overhead, they developed a lightweight, specific protocol. This protocol is designed to minimize latency and maximize throughput for bitstream transfers.
  • Hardware/Software Co-design: The authors emphasize a balanced partition between hardware and software. They designed a system that leverages dedicated hardware blocks (like DMAs) for efficient data movement and a streamlined software architecture to manage the data pipeline from the Ethernet controller to the FPGA's Internal Configuration Access Port (ICAP).
  • Producer-Consumer Paradigm: A software producer-consumer model is employed, where the Ethernet controller acts as the producer, feeding data into a circular buffer, and the ICAP acts as the consumer, receiving data from the buffer. This asynchronous approach helps decouple the network reception from the FPGA configuration process.
  • DMA Utilization: The use of Direct Memory Access (DMA) controllers is crucial for offloading data transfer tasks from the main processor, allowing for higher speeds and reducing software overhead. They implemented two DMAs: one for transferring data from the Ethernet controller to a buffer and another for transferring data from the buffer to the ICAP.
  • Optimized Buffer Management: The size and management of the circular buffer are critical. The paper highlights that their protocol dynamically adapts burst sizes to the buffer, and that a relatively small buffer (requiring only a few packets) is sufficient for achieving high speeds.

Datasets

Not discussed in the paper. The evaluation uses synthesized cryptographic IPs (DES and triple DES) to generate partial bitstreams of varying sizes (60 KB and 200 KB) for testing.

Key Insights and Ideas

  • Standard Ethernet is Capable of High-Speed PR: The paper demonstrates that with the right protocol and architecture, standard Ethernet (even 100 Mb/s) can be a surprisingly fast conduit for FPGA reconfiguration, far exceeding the performance of traditional network stacks.
  • Protocol Overhead is the Real Bottleneck: The significant speedup achieved is attributed to bypassing the overhead of standard TCP/IP and other complex network protocols, highlighting that the "network" itself isn't the limit, but rather how we use it for this specific task.
  • Minimal On-Chip Memory is Sufficient: A key takeaway is that achieving high PR speeds does not necessitate large amounts of on-chip memory for bitstream storage. Their approach requires only tens of KB, making it suitable for cost-sensitive and resource-constrained embedded systems.
  • Hardware-Software Partitioning is Crucial: The evolution from their first architecture (Arch-1) to the second (Arch-2) shows how optimizing the hardware-software interface, particularly through the use of DMAs, can dramatically improve performance.
  • "Figure of Merit" (FOM) is a Useful Comparison Tool: The paper effectively uses the FOM (bits per second per MHz) to normalize performance across different system frequencies, allowing for fair comparison with other works.

Key Results

The paper reports significant improvements in partial reconfiguration speeds:

  • Ultra-Fast Downloading: Their optimized architecture (Arch-2) achieves a sustained download speed of 80 Mb/s over a 100 Mb/s Ethernet link.
  • High FOM: The achieved Figure of Merit (FOM) is approximately 800 Kb/(s.MHz), which is stated to be 20 times faster than previous related works.
  • Minimal Memory Footprint: The software implementation requires a small memory footprint, around 32 KB for data memory and 40 KB for executable code, fitting within the FPGA's internal memory resources.
  • Small Buffer Requirements: The producer-consumer buffer requires room for only about six packets, totaling approximately 9 KB, which is significantly less than typical buffer pools used in standard protocol stacks.

Limitations and Future Work

  • Limitations:
    • The current work focuses on 100 Mb/s Ethernet. While the FOM is frequency-independent, higher bandwidth Ethernet (e.g., 1 Gb/s) could potentially yield even higher absolute speeds.
  • The current ICAP interface is 8 bits wide. Wider ICAP interfaces (16 or 32 bits) could further increase the FOM.
  • The current implementation is tested on Virtex-2 and Virtex-4 FPGAs. Testing on newer architectures like Virtex-5 is planned.
  • Future Work:
    • Higher Bandwidth Ethernet: Testing with 1 Gb/s Ethernet to explore higher absolute download speeds.
  • Wider ICAP Interfaces: Investigating the use of 16-bit and 32-bit ICAP word accesses to further boost the FOM.
  • Integration with Standard Stacks: Exploring integration of their protocol on top of UDP within existing stacks like lwIP or within real-time operating systems (Linux, MicroC/OS-II) to promote wider adoption.
  • Alternative Architectures: Developing software implementations for Microblaze soft cores and full hardware implementations for systems without PPC405 hard cores.
  • Other Network Interfaces: Adapting the protocol for other network types like Wi-Fi or CAN.

Summary updated

BibTeX
@inproceedings{bomel2009ultra,
  author = {Pierre Bomel and Jérémie Crenne and Linfeng Ye and Jean-Philippe Diguet and Guy Gogniat},
  title = {{Ultra-Fast Downloading of Partial Bitstreams through Ethernet}},
  booktitle = {ARCS},
  year = {2009},
  doi = {10.1007/978-3-642-00454-4\_10},
}
Figure