TL;DR
This paper introduces SFIP (Syscall-Flow-Integrity Protection), a novel security mechanism that extends Control-Flow Integrity (CFI) to the user-kernel boundary. By analyzing application code to define valid syscall sequences and their origins, SFIP significantly reduces the attack surface and prevents sophisticated exploits with minimal performance overhead.
What Problem Does This Paper Address?
Modern applications are complex, leading to an increasing number of vulnerabilities. While Control-Flow Integrity (CFI) has been effective in preventing attacks within user-space applications, it doesn't protect the critical interface between user-space and the kernel. Attackers who bypass user-space CFI can then exploit vulnerabilities in the system or its interfaces to gain elevated privileges. Existing sandboxing techniques like seccomp primarily focus on allowing or denying individual syscalls, but they lack the context of valid syscall sequences. This leaves a gap where attackers can still chain together allowed syscalls in malicious ways. The paper addresses this gap by proposing a system that enforces not just which syscalls are allowed, but also in what order and from where they can be invoked.
Methods
SFIP is built upon three core pillars:
- Syscall State Machine: This pillar models valid transitions between consecutive syscalls. Instead of individual sequences, it uses an efficient N×N matrix representation, where N is the number of available syscalls. This allows for O(1) lookup of valid transitions.
- Syscall-Origin Mapping: This component maps each syscall to the specific locations within the application's code from which it is allowed to be invoked. This addresses the challenge that a single syscall instruction might be used for multiple syscalls, or a single syscall number might be invoked from various locations.
- Kernel Enforcement: A modified Linux kernel enforces the rules defined by the state machine and origin mapping. If a syscall violates these rules, the application is immediately terminated.
The system uses a compiler-based approach (SysFlow, built on LLVM) to automatically extract the necessary information (syscall state machine and origin mapping) from application source code during compilation. This information is then embedded into the binary and installed in the kernel at runtime.
Datasets
The paper evaluates SFIP on several real-world applications, including:
- nginx
- ffmpeg
- mupdf
- busybox
- coreutils
- mutool
These applications were chosen to represent a range of sizes and complexities, and have been used in prior research on automated seccomp filter generation. No new datasets were introduced.
Key Insights and Ideas
- Extending CFI to the Kernel Boundary: The core idea is to apply the principles of CFI (valid sequences and origins) to the user-kernel syscall interface, a critical but often unprotected area.
- Syscall Sequences Matter: Simply allowing individual syscalls (like seccomp) is insufficient. The order and context of syscalls are crucial for security, as attackers can chain allowed syscalls to achieve malicious goals.
- Origin Matters: Restricting syscalls not only by their type but also by their invocation location significantly shrinks the attack surface, especially against code-reuse attacks like ROP.
- Automation is Key: Manually defining these rules is impractical for modern applications. SFIP's strength lies in its automated extraction of state machines and origin mappings from source code.
- Performance is Feasible: The proposed system achieves its security goals with minimal performance overhead, making it practical for real-world deployment.
Key Results
- Performance:
- Microbenchmark: SFIP introduces an overhead of 13.1% for combined state machine and origin enforcement. This is comparable to or better than seccomp (15.23%).
- Macrobenchmark (nginx, ffmpeg): Overhead was minimal, around 1.8% for combined enforcement.
- Security:
- SFIP significantly reduces the number of possible syscall transitions. On average, it decreases possible transitions by 38.6% compared to seccomp and 90.9% compared to no protection.
- Demonstrated effectiveness against control-flow hijacking attacks, including preventing an exploit in BusyBox by blocking invalid syscall transitions (e.g., from
sockettoexecve). - Analysis of real-world applications showed that SFIP drastically limits the number of valid syscall origins for specific syscalls (e.g., reducing possible origins for a syscall in nginx from 318 to 3).
- Extraction Time: The automated extraction process introduces a significant compilation time overhead (e.g., ~28x for nginx), but this is a one-time cost per application version.
Limitations and Future Work
- Signal Handlers: The current proof-of-concept does not handle signals and syscalls invoked within signal handlers. The authors suggest extending SFIP with per-signal state machines.
- Fine-Grained CFI: The current state machines are coarse-grained. Future work could integrate syscall origin information more deeply into the state machine for finer-grained control, making transitions dependent not only on the previous syscall but also its location.
- Per-Thread State Machines: Similar to per-signal state machines, per-thread state machines could offer improved security.
- Dynamic Analysis: While the current approach is static, future work could explore incorporating dynamic analysis or machine learning techniques for more precise models.
- Kernel Updates: The current kernel enforcement does not allow updates to installed SFIP information. While this enhances security, future implementations might consider controlled updates.
Summary updated