Abstract
Traditional software behavior models often struggle to balance detection accuracy with computational efficiency. This paper introduces a novel approach using "functional slicing" to dynamically analyze system calls. By categorizing system calls and applying specific slicing rules, the model focuses on high-threat behaviors and their arguments while filtering out background noise. Experimental evaluations using the sendmail benchmark demonstrate that this method achieves superior accuracy (lower Mean Absolute Deviation) and significantly better runtime efficiency compared to established models like HPDA and HFA.
Introduction
In the field of cybersecurity, monitoring a program's system calls is a standard technique for building "normal" behavior profiles. Deviations from these profiles can signal malicious exploits. However, existing models—such as Finite State Automata (FSA) and Pushdown Automata (PDA)—often face a "Goldilocks" problem: they are either too simple to capture complex execution paths, leading to false negatives, or so complex that they create significant performance bottlenecks. This paper addresses this by narrowing the monitoring focus to specific "slices" of program execution.
Methodology
The authors propose a Software Behavior Automaton Model based on a 5-tuple $(P, S, E_0, E_S, S_T)$, where $P$ represents function slices and $S$ represents the running state (defined as a combination of key system calls and functions).
System Call Classification
A core component of this model is the classification of system calls into five functional groups: File System, Process, Network, Module, and Signal. Crucially, these calls are further divided into Start calls, System calls, and End calls. This distinction allows the model to identify the logical boundaries of specific functions and filter out irrelevant operations.
Functional Slicing Rules
The model applies five specific rules to restrict monitoring. For instance, a rule might specify that the system should only record calls affecting a particular resource or function at a specific point in the program. Using the strace utility, the algorithm monitors the running program and records only those calls that match both the high-threat classification and the defined slicing rules.
Experimental Results
The researchers evaluated the model using the "New Mexico University sendmail sample," a standard benchmark in the intrusion detection community. The proposed model was compared against the High-level Pushdown Automaton (HPDA) and the Hybrid Automaton (HFA).
Accuracy and Efficiency
The model's accuracy was assessed using Mean Absolute Deviation (MAD), which measures the difference between the forecast model and the ground truth. As shown in the results below, the proposed model consistently maintained a lower MAD (approximately 0.13) compared to HPDA (0.14–0.18) across 300 states.
Furthermore, the runtime efficiency of the proposed model scaled much better than its competitors. At 10 states, the proposed model's runtime was approximately 4ms, while HFA and HPDA reached roughly 6.5ms and 8.5ms, respectively.
Conclusion
The functional slicing approach provides a robust method for ensuring software trustworthiness by reducing the state space and focusing on security-critical behaviors. While future work will address the overhead of tools like strace and the automation of slicing rules, the current results demonstrate a clear path forward for efficient, real-time anomaly detection in complex software systems.
Summary updated