TL;DR
This paper introduces CUTBUF, a novel Network-on-Chip (NoC) router architecture that dynamically reuses buffers and virtual channels (VCs) across different virtual networks (VNETs). This approach significantly reduces the number of physical buffers required in routers, leading to substantial savings in area and power consumption, while maintaining comparable performance to traditional designs.
What Problem Does This Paper Address?
Modern multi-core processors rely on Networks-on-Chip (NoCs) to handle increasing communication demands. A critical component of these NoCs is the router, and its buffer design significantly impacts performance, area, and power consumption. Coherence protocols, common in Chip Multiprocessor (CMP) systems, often require multiple Virtual Networks (VNETs) to prevent deadlocks. This necessitates dedicating separate sets of Virtual Channels (VCs) and their associated buffers to each VNET, leading to underutilization of buffer resources, especially when traffic is unbalanced across VNETs. This underutilization translates to wasted area and power. The challenge is to design a router that can efficiently manage buffers to reduce area and power without sacrificing performance, while still adhering to deadlock-free requirements for both network and protocol levels.
Methods
CUTBUF introduces several key innovations:
- Dynamic VC Allocation to VNETs: Instead of statically assigning VCs to specific VNETs, CUTBUF allows any VC to be dynamically allocated to any VNET based on real-time traffic demands. This flexibility helps balance resource utilization.
- Traffic Mixing and Queue Usage Minimization: CUTBUF enables different types of messages (from different VNETs) to share the same VC at different times. This "traffic mixing" significantly increases buffer utilization by consolidating traffic into fewer physical buffers.
- Buffer Reuse: This strategy allows a buffer to be reused by a new packet as soon as the previous packet stored in it is guaranteed to exit the router within a finite number of cycles. This is achieved by checking for tail flit reception and downstream buffer availability, ensuring no deadlocks are introduced.
- Virtual Network (VNET) Reuse: This optimization allows packets from different VNETs to share VCs, further increasing buffer utilization. It ensures that at least one VC per VNET is always available to prevent protocol-level deadlocks.
- Switch Allocation Flow (SAF) and Release Blocked Crossbar Path (RBCP): These techniques optimize crossbar allocation by prioritizing packets that have already won the SA stage, reducing contention and improving throughput. RBCP further enhances this by releasing blocked paths when downstream buffers are available.
- Buffer Remapping: This logic allows for a "late binding" between a packet and its physical buffer. It ensures that even if a VC is assigned upstream, the buffer remapping logic can select the most appropriate physical buffer downstream, transparently to the upstream router.
Datasets
The paper evaluates CUTBUF using:
- Synthetic Traffic: Random, bit-complement, and transpose traffic patterns are used to stress the network under various load conditions.
- Real Benchmarks:
- MIBENCH: A suite of embedded benchmark applications.
- PARSEC: A benchmark suite for parallel applications, used for scalability analysis up to 48 cores.
- Simulation Environment: GEM5 full-system simulator is used for evaluation.
Key Insights and Ideas
- Buffer Underutilization is a Major Bottleneck: The paper strongly emphasizes that buffers are the dominant factor in NoC router area and power. The inherent imbalance in VNET traffic leads to significant underutilization, which CUTBUF directly addresses.
- Dynamic Resource Allocation is Key: The core idea is that static allocation of resources (VCs and buffers) to VNETs is inefficient. Dynamic allocation based on actual traffic needs is crucial for optimizing performance, area, and power.
- Deadlock-Free Reuse is Achievable: A significant contribution is demonstrating that buffer and VC reuse, which could intuitively lead to deadlocks, can be implemented in a deadlock-free manner by carefully considering packet state and downstream buffer availability.
- Performance Trade-offs Can Be Managed: While CUTBUF aims for buffer reduction, it doesn't necessarily mean a performance hit. By intelligently reusing resources and employing optimized allocation strategies, CUTBUF can achieve comparable or even better performance than baseline designs, especially when buffer resources are limited.
Key Results
- Buffer Reduction: CUTBUF can reduce the number of physical buffers by up to 33%.
- Area and Power Savings: Achieves area and power savings of up to 30.6% and 30.7%, respectively, compared to a baseline with more buffers.
- Performance Impact:
- With fewer buffers, performance degradation is typically less than 5%, often below 2% for real applications.
- When using the same number of buffers as the baseline, CUTBUF can improve performance by up to 23.8%.
- Frequency Decrease: A minor operating frequency decrease of around 5% was observed.
- Overhead: Area and power overheads are generally low, especially with larger flit sizes and deeper buffers, and are largely dominated by the buffer component itself.
Limitations and Future Work
- Power Gating Not Explored: The paper mentions that power gating actuators could offer further power savings, but this aspect was not explored in the current work.
- Out-of-Order Issues: VNET reuse might introduce out-of-order packet delivery, a known issue also present in baseline multi-VC designs.
- SAF Optimization: While SAF is beneficial, the paper notes that it can lead to Head Induced Block (HIB) or Tail Induced Block (TIB), which are mitigated by the RBCP optimization.
- Future Work: The authors suggest exploring power gating for further savings and investigating the impact of CUTBUF on different NoC topologies and routing algorithms. The flexibility of CUTBUF to adapt to various coherence protocols and traffic patterns is highlighted as a key advantage for future research.
Summary updated