Cache Where you Want! Reconciling Predictability and Coherent Caching

Ayoosh Bansal, Jayati Singh, Yifan Hao, Jen-Yang Wen, Renato Mancuso, Marco Caccamo

Abstract

Real-time and cyber-physical systems need to interact with and respond to their physical environment in a predictable time. While multicore platforms provide incredible computational power and throughput, they also introduce new sources of unpredictability. Large fluctuations in latency to access data shared between multiple cores is an important contributor to the overall execution-time variability. In addition to the temporal unpredictability introduced by caching, parallel applications with data shared across multiple cores also pay additional latency overheads due to data coherence. Analyzing the impact of data coherence on the worst-case execution-time of real-time applications is challenging because only scarce implementation details are revealed by manufacturers. This paper presents application level control for caching data at different levels of the cache hierarchy. The rationale is that by caching data only in shared cache it is possible to bypass private caches. The access latency to data present in caches becomes independent of its coherence state. We evaluate the system on an architectural simulator and show that the worst-case execution time for a single memory write request is reduced by 52%. Benchmark evaluations show that the proposed technique has a minimal impact on average performance.

The Challenge of Predictable Coherence

In real-time systems, such as autonomous vehicles or industrial robotics, "average speed" is secondary to "guaranteed timing." Engineers must calculate the Worst-Case Execution Time (WCET) to ensure safety-critical responses always occur within strict deadlines. However, modern multi-core processors introduce significant unpredictability through cache coherence protocols.

When multiple cores share data, hardware controllers (typically using the MSI protocol) must coordinate to ensure data consistency. These controllers are often proprietary "black boxes." A simple memory write can vary from a few cycles to hundreds if it triggers a "dirty miss"—a scenario where data must be invalidated in one core's private cache and fetched from another.

The complex sequence of events in a "dirty miss" coherence transaction, illustrating the multi-step process that leads to high latency.

This massive variability forces engineers to assume the worst-case scenario, leading to vastly underutilized hardware.

Proposed Solution: INC-OC Memory

The authors propose a hardware/software co-design approach to achieve "predictable coherence" by introducing a new memory attribute: Inner-Non-Cacheable, Outer-Cacheable (INC-OC).

  1. Bypassing Private Caches: Data marked as INC-OC is ignored by private L1 caches (the "Inner" level) but allowed to reside in shared L2 or L3 caches (the "Outer" level).
  2. Trivial Coherence: By ensuring shared data exists in only one location—the shared cache—multiple copies are never created. This bypasses the complex hardware coherence protocols entirely, making coherence "trivially satisfied."
  3. Implementation: The researchers modified the Linux kernel to support a new flag in the mmap system call, allowing developers to explicitly mark shared buffers as INC-OC. They also used the gem5 architectural simulator to modify L1 cache controllers to forward INC-OC requests directly to the L2.

Evaluation and Key Results

The system was evaluated using synthetic microbenchmarks designed to trigger worst-case "dirty miss" scenarios and the SPLASH-2 benchmark suite for real-world performance analysis.

The results demonstrate significant improvements in predictability and performance for shared data:

  • Latency Reduction: In simulated "dirty miss" scenarios, the latency for a single write request was reduced by 52% (from 729 cycles to 354 cycles).
  • Locking Efficiency: Surprisingly, using INC-OC for spinlocks significantly reduces acquisition time. On a simulated Cortex-A53, shared access latency for locks dropped from 363ns to 116ns because the "handshake" between cores occurs at the L2 level rather than through a series of L1 invalidations.
Comparison of access latencies for private vs. shared data across different operations and architectures, highlighting the dramatic reduction in lock latency.
  • Average Performance: When using a "Program Aware" approach—applying INC-OC only to specific shared variables—the performance impact on the SPLASH-2 suite was negligible. INC-OC accesses accounted for less than 0.08% of total memory traffic while providing the necessary timing guarantees.

Conclusion and Future Work

The INC-OC approach offers an elegant solution to the unpredictability of cache coherence in multi-core real-time systems. By trading the speed of L1 for the certainty of L2 for shared data, it significantly reduces WCET.

Future work includes automating the identification of INC-OC variables via compiler analysis. While the ARMv8-A architecture theoretically supports such controls, widespread adoption depends on hardware manufacturers fully implementing independent inner/outer cacheability bits in physical silicon.

Summary updated

Abstract

Real-time and cyber-physical systems need to interact with and respond to their physical environment in a predictable time. While multicore platforms provide incredible computational power and throughput, they also introduce new sources of unpredictability. Large fluctuations in latency to access data shared between multiple cores is an important contributor to the overall execution-time variability. In addition to the temporal unpredictability introduced by caching, parallel applications with data shared across multiple cores also pay additional latency overheads due to data coherence. Analyzing the impact of data coherence on the worst-case execution-time of real-time applications is challenging because only scarce implementation details are revealed by manufacturers. This paper presents application level control for caching data at different levels of the cache hierarchy. The rationale is that by caching data only in shared cache it is possible to bypass private caches. The access latency to data present in caches becomes independent of its coherence state. We discuss the existing architectural support as well as the required hardware and OS modifications to support the proposed cacheability control. We evaluate the system on an architectural simulator. We show that the worst case execution time for a single memory write request is reduced by 52%. Benchmark evaluations show that proposed technique has a minimal impact on average performance.
BibTeX
@inproceedings{bansal2020cache,
  author = {Ayoosh Bansal and Jayati Singh and Yifan Hao and Jen-Yang Wen and Renato Mancuso and Marco Caccamo},
  title = {{Cache Where you Want! Reconciling Predictability and Coherent Caching}},
  booktitle = {MECO},
  year = {2020},
  url = {https://openreview.net/forum?id=RNELCRV9ViI},
}
Figure