Abstract
Memory safety vulnerabilities remain a persistent threat in systems programmed with low-level languages like C and C++. While hardware-supported security extensions, such as Intel Control-flow Enforcement Technology (CET) and Arm Memory Tagging Extension (MTE), offer promising low-overhead defenses, their effectiveness across diverse platforms remains difficult to quantify. This paper introduces an open-source, expandable framework and a comprehensive test suite of 160 cases designed to evaluate memory safety defenses across various processor architectures, operating systems, and compilers. By utilizing a novel relation graph to map vulnerability dependencies, the framework provides a standardized methodology for measuring security coverage and identifying gaps in modern hardware-software stacks.
Introduction
Memory safety remains the "eternal war" of computer security. Low-level languages like C and C++ are prone to bugs—such as buffer overflows or use-after-free errors—that allow attackers to hijack systems. While many software-based defenses exist, they often impose a heavy performance penalty, leading developers to disable them in production.
Recently, chipmakers have introduced hardware-supported defenses to provide security with minimal overhead. However, there hasn't been a consistent way to test if these features actually work as advertised across different platforms. Existing test suites are often outdated, limited to specific architectures (like x86), or fail to distinguish which specific part of the system (the hardware, the kernel, or the compiler) is actually providing the protection. This work addresses these gaps by providing a portable, comprehensive, and logically structured testing framework.
Methodology
The authors' framework is built on three main pillars: a logical relation graph, a cross-platform architecture, and anti-optimization techniques.
The Relation Graph
Instead of treating vulnerabilities as a flat list, the authors created a graph that defines the relationships between them. This includes dependencies (e.g., a Return-Oriented Programming (ROP) attack might depend on bypassing ASLR), specialization, and relaxation. This allows the suite to determine "coverage" logically—if a "prerequisite" defense fails, the suite knows which subsequent attacks are likely to succeed.
Cross-Platform Architecture and Portability
To ensure the suite works on x86, Arm, and RISC-V, the authors separated the tests into platform-independent C++ code and a small, shared library of platform-specific assembly macros. This modularity allows the framework to be easily extended to new architectures as they emerge.
Anti-Optimization Techniques
Compilers are often "too smart" for security testing; they frequently optimize away the intentional bugs used in test cases. The authors used specific compiler attributes, such as volatile variables and noinline functions, alongside embedded assembly to ensure the vulnerabilities remain exploitable for testing purposes.
The Memory Safety Test Suite
The primary contribution is a suite of 160 specialized test cases, categorized into five key areas:
- Spatial Safety (98 cases): Covers buffer overflows and underflows across the stack, heap, and global data.
- Temporal Safety (13 cases): Focuses on use-after-free and double-free vulnerabilities.
- Access Control (3 cases): Evaluates the effectiveness of ASLR and page permissions.
- Pointer Integrity (5 cases): Tests protections for function pointers and VTables.
- Control-Flow Integrity (41 cases): Targets ROP, Jump-Oriented Programming (JOP), and VTable injection.
The suite was evaluated on six distinct platforms, including Intel i7/Xeon, Nvidia Jetson Nano (Arm), Raspberry Pi 4B (Arm), and HiFive Unleashed (RISC-V).
Key Insights and Results
The evaluation revealed several critical insights into the current state of hardware-supported security:
- The "Default" Vulnerability: On almost all tested platforms, the "default" configuration provides very little protection. Many hardware features are present but remain unused because the OS or compiler hasn't enabled them. Across all six platforms, zero defenses were detected by default against spatial safety attacks.
- RISC-V Maturity: The RISC-V platform (using the Spike simulator) showed marginal security benefits due to its support for "Read-Exclusive Execution" (R⊕X) in page tables, which thwarts certain code-injection attacks.
- ASan's Dominance: While hardware features are the future, the software-based AddressSanitizer (ASan) remains the most effective defense in the suite, catching nearly all spatial and temporal bugs, though it is usually too slow for production use.
- Compiler Flags: Enabling the
RELRO(Relocation Read-Only) flag successfully protected the Global Offset Table (GOT) on most platforms, but other flags likeStack Protectionprovided very limited defense against sophisticated ROP attacks.
Limitations and Future Work
While comprehensive, the framework has room for growth. Current limitations include a lack of support for multi-threaded concurrency vulnerabilities and Data-Oriented Programming (DOP) attacks. Furthermore, the framework does not yet test for side-channel or transient execution attacks (like Spectre or Meltdown), though it is designed to be expanded in these directions. Future iterations aim to move from binary pass/fail results to a more nuanced quantitative security scoring system.
Summary updated