TL;DR
This paper introduces novel Level-0 (L0) cache designs for embedded processors that improve performance and reduce energy consumption by intelligently managing data movement between L0 and L1 caches. The proposed "flow cache" and "hit cache" policies offer significant benefits, particularly for direct-mapped and set-associative L1 caches, respectively, by reducing conflict misses and filtering L1 accesses.
What Problem Does This Paper Address?
Embedded processors, often resource-constrained, benefit from performance and energy optimizations. Level-0 (L0) caches, small caches placed before the Level-1 (L1) cache, have been proposed to improve performance and reduce energy. However, the effectiveness of L0 caches hinges on efficient data migration policies between L0 and L1. Existing approaches often assume L0 access precedes L1, potentially increasing latency, or lack a systematic study of migration strategies. This paper addresses this gap by proposing and evaluating new L0 cache organizations and migration policies that assume L0 hit/miss determination can be made before L1 access, enabling more flexible and efficient data management.
Methods
The authors propose two novel L0 cache organizations and associated migration policies:
- Flow Cache (FC): Designed for direct-mapped L1 caches, this policy aims to reduce conflict misses. When a miss occurs, the new line is first inserted into the L0 cache. If the L0 cache evicts a line, it's then migrated to the L1 cache. On an L1 hit, the line's state is updated, but no data movement occurs between caches. This design prioritizes simpler hardware and lower dynamic energy compared to victim caches.
- Hit Cache (HC): Tailored for set-associative L1 caches, this policy focuses on reducing L1 access energy by filtering accesses. On an L1 miss, the line is inserted into the L1 cache. On an L1 hit, the line is promoted to the L0 cache, and the evicted L0 line is written back to L1. The paper explores two variants: "eager" (write-through to L1 on L0 write) and "lazy" (L1 copy invalidated on L0 write, writeback delayed). The lazy approach offers better energy savings but a slight performance degradation.
The core of their approach involves systematically studying various insertion and promotion policies for data migration between L0 and L1 caches, analyzing their impact on performance (misses per kilo instruction - MPKI) and energy consumption. They also propose a processor-cache interface that decouples L0 tag and data arrays, allowing L0 tag comparison to occur early in the pipeline.
Datasets
The authors evaluated their proposed policies using benchmarks from the MiBench, CommBench, and MediaBench suites. They compiled these benchmarks using a cross-compiler for the Alpha architecture and ran them for 200 million instructions or until completion. The paper does not introduce new benchmarks but uses a representative set of existing embedded system workloads.
Key Insights and Ideas
- Early L0 Hit/Miss Determination is Key: The assumption that L0 hit/miss can be determined before L1 access is crucial. This allows for more flexible migration policies and avoids increasing L1 access latency.
- L0 Cache Specialization: Different L0 cache designs are optimal for different L1 cache configurations. Flow caches are effective for direct-mapped L1s (reducing conflict misses), while Hit caches are better for set-associative L1s (reducing energy).
- Flow Cache Simplicity and Efficiency: The flow cache offers comparable conflict miss reduction to victim caches but with simpler hardware and lower dynamic energy consumption, making it attractive for embedded systems.
- Hit Cache Energy Savings: The hit cache significantly reduces L1 access energy by filtering accesses, especially with the "lazy" update policy, which can achieve up to 50% energy reduction at the cost of a small performance hit.
- Reuse Distance Matters: Analysis of reuse distance distributions confirms that small L0 caches can be beneficial for lines with short reuse distances, justifying their use for filtering and buffering.
Key Results
- Flow Cache: Achieves similar performance to victim caches in reducing D-cache misses for direct-mapped L1s, with lower dynamic energy consumption. For example, a 2-way flow cache on a 16KB L1 cache removes 20% misses with only a 6% energy increase, whereas a victim cache with the same configuration increases energy by 27%.
- Hit Cache:
- The "eager" design offers similar performance to the baseline L1 cache with energy savings of around 30%.
- The "lazy" design achieves up to 50% energy savings but with a 5% performance degradation.
- L0 hit rates for hit caches are high (60%+), indicating effective filtering of L1 accesses.
Limitations and Future Work
- Performance vs. Energy Trade-off: While the "lazy" hit cache offers significant energy savings, it comes with a slight performance penalty (5% more misses). The paper suggests that the optimal choice depends on the specific application's priorities.
- Hardware Design Complexity: The paper discusses hardware design aspects but acknowledges that detailed implementation and optimization for specific embedded architectures would be necessary.
- Multi-core Environment: The paper briefly touches upon multi-core implications, suggesting that policies with inclusiveness are more transparent. Further research in multi-core scenarios could be beneficial.
- Simulation-based Evaluation: The results are based on simulation using the gem5 simulator and Cacti for energy estimation. Real-world hardware validation would be the next step.
Summary updated