Designing Mixed Criticality Applications on Modern Heterogeneous MPSoC Platforms

Giovani Gracioli, Rohan Tabish, Renato Mancuso, Reza Mirosanlou, Rodolfo Pellizzoni, Marco CaccamoECRTS 2019 · OpenReview

TL;DR

This paper presents a novel hardware-software co-design approach for heterogeneous MPSoCs to enable mixed-criticality real-time systems. By leveraging programmable logic (PL) for dedicated scratchpad memories (SPMs) and a custom address translator, they achieve strong isolation between critical and non-critical tasks, significantly improving predictability and performance under contention.

What Problem Does This Paper Address?

Modern embedded systems are increasingly demanding, requiring the simultaneous execution of high-performance, soft real-time applications alongside strict, hard real-time tasks. Heterogeneous MPSoCs, combining traditional CPU cores with programmable logic (PL) like FPGAs, offer the potential to meet these diverse needs. However, a major challenge is ensuring that the resource-intensive activities of soft real-time tasks (e.g., heavy memory access) do not interfere with the predictable execution of critical real-time tasks. Existing solutions often struggle with issues like shared resource contention (especially in the Last-Level Cache - LLC) and memory waste when implementing isolation mechanisms. This paper aims to bridge this gap by providing a practical, full-stack solution for mixed-criticality systems on these advanced MPSoCs.

Methods

The authors propose a multi-faceted approach:

  • Criticality Domains: The system is partitioned into distinct "criticality domains" (low, mid, and high), each assigned to specific cores and resources. Strong isolation is enforced between domains, particularly from low-criticality to high-criticality.
  • Hardware Isolation with PL:
    • Dedicated SPMs: Programmable logic (PL) is used to implement private Scratchpad Memories (SPMs) for mid- and high-criticality domains. These SPMs provide fast, predictable local storage, reducing reliance on shared main memory.
  • Address Translator: A custom hardware IP acts as an "address translator" between the CPU subsystem (PS) and the PL. This component is crucial for mitigating memory waste caused by cache coloring. It intelligently maps colored addresses to contiguous memory locations within the SPMs, preserving the effectiveness of coloring without sacrificing memory capacity.
  • Software Techniques:
    • Jailhouse Hypervisor: A partitioning hypervisor (Jailhouse) is employed to manage resource isolation. It handles core partitioning, LLC partitioning via page coloring, and facilitates inter-domain communication.
  • Page Coloring: Used for LLC partitioning to enforce spatial isolation between cores.
  • Three-Phase Model: Real-time tasks utilize a three-phase execution model (load, execute, unload) to leverage SPMs, managed by a DMA engine.
  • Erika RTOS: A real-time operating system (RTOS) is used on the critical cores to manage task scheduling and execution.

Datasets

The paper uses the following datasets/benchmarks for evaluation:

  • San Diego Visual Benchmark Suite (SD-VBS): Specifically, the disparity and mser applications are used. These are described as representative of real-time applications found in areas like autonomous driving, requiring both predictability and performance.
  • Bandwidth Benchmark (BW): A custom benchmark designed to stress the memory subsystem by issuing writes to DRAM or SPM, ensuring cache misses. This is used to simulate contention.
  • KITTI Vision Benchmark Suite dataset: Used for the image processing case study, specifically 20 frames from the 2015 stereo multiview dataset, which were then downscaled.

Key Insights and Ideas

  • PL as a Predictability Enabler: The paper strongly demonstrates that programmable logic isn't just for acceleration; it's a powerful tool for building predictable real-time systems by providing dedicated, isolated resources like SPMs.
  • Cache Coloring Without Waste: The custom address translator is a key innovation. It elegantly solves the problem of memory waste associated with cache coloring on small, critical memories like SPMs, which is a significant practical hurdle.
  • Full-Stack Integration: The success hinges on the seamless integration of hardware (PL, translator) and software (hypervisor, RTOS, custom drivers). This holistic approach is crucial for achieving the desired isolation and performance.
  • Performance vs. Isolation Trade-off: The results show that while dedicated SPMs might be slightly slower than PS DRAM, the isolation they provide, especially when combined with cache coloring and the address translator, leads to significantly better predictability and often comparable or even improved performance under contention.

Key Results

The paper presents results from experiments on a Xilinx UltraScale+ ZCU102 MPSoC, evaluating various scenarios:

  • Contention Impact: In legacy scenarios without isolation, contention from memory-intensive tasks caused significant increases in Worst-Case Execution Time (WCET) and variability. For mser, WCET increased by 1.73x and variability by 27.8% in a stressed scenario (LCY-STRESS) compared to solo execution (LCY-SOLO).
  • Proposed Solution Effectiveness:
    • mser application: In the high-criticality domain (OUR-HIGH), the variability window was reduced by 58% and WCET by 37% compared to the stressed legacy scenario. Performance was close to the ideal solo execution (OUR-SOLO).
  • disparity application: In the high-criticality domain (OUR-HIGH), WCET was minimally affected by contention, with a 1.06x increase compared to solo execution (OUR-SOLO). Variability was significantly reduced.
  • Sustainable Frequencies: For the image processing case study (disparity), the proposed system allowed for higher sustainable processing frequencies under load. For 64x48 images, the sustainable rate increased from 55 Hz (LCY-STRESS) to 59 Hz (OUR-HIGH). For 128x96 images, the sustainable rate was 14 Hz (OUR-HIGH) compared to 12 Hz (LCY-STRESS).

Limitations and Future Work

  • Limitations:
    • The current implementation of inter-cell communication relies on shared memory, with future work planned to enhance this with a server-based scheduling mechanism for IPIs.
  • The analysis of DMA programming overhead's impact on schedulability is deferred to future work.
  • The security of the communication engine is a concern; failure of this component could bring down the entire system.
  • Future Work:
    • Enhance Jailhouse with a server-based scheduling mechanism for inter-processor interrupts (IPIs) between different criticality levels.
  • Develop compiler code generation for automatically creating load/unload requests for tasks using the three-phase model.
  • Provide a formal schedulability analysis for the fine-granularity TDMA-based DMA.
  • Investigate methods to guarantee the safety of the communication engine, especially against security attacks.
  • Explore achieving higher computational power for image processing applications without sacrificing isolation.

Summary updated

Abstract

Multiprocessor Systems-on-Chip (MPSoC) integrating hard processing cores with programmable logic (PL) are becoming increasingly common. While these platforms have been originally designed for high performance computing applications, their rich feature set can be exploited to efficiently implement mixed criticality domains serving both critical hard real-time tasks, as well as soft real-time tasks. In this paper, we take a deep look at commercially available heterogeneous MPSoCs that incorporate PL and a multicore processor. We show how one can tailor these processors to support a mixed criticality system, where cores are strictly isolated to avoid contention on shared resources such as Last-Level Cache (LLC) and main memory. In order to avoid conflicts in last-level cache, we propose the use of cache coloring, implemented in the Jailhouse hypervisor. In addition, we employ ScratchPad Memory (SPM) inside the PL to support a multi-phase execution model for real-time tasks that avoids conflicts in shared memory. We provide a full-stack, working implementation on a latest-generation MPSoC platform, and show results based on both a set of data intensive tasks, as well as a case study based on an image processing benchmark application.
BibTeX
@inproceedings{gracioli2019designing,
  author = {Giovani Gracioli and Rohan Tabish and Renato Mancuso and Reza Mirosanlou and Rodolfo Pellizzoni and Marco Caccamo},
  title = {{Designing Mixed Criticality Applications on Modern Heterogeneous MPSoC Platforms}},
  booktitle = {ECRTS},
  year = {2019},
  url = {https://openreview.net/forum?id=SX6gLeJkBze},
}
Figure