Abstract
Software obfuscation techniques can prevent binary diffing techniques from locating vulnerable code by obfuscating the third-party code, to achieve the purpose of protecting embedded device software. With the rapid development of binary diffing techniques, they can achieve more and more accurate function matching and identification by extracting the features within the function. This makes existing software obfuscation techniques, which mainly focus on the intra-procedural code obfuscation, no longer effective. In this paper, we propose a new inter-procedural code obfuscation mechanism Khaos, which moves the code across functions to obfuscate the function by using compilation optimizations. Two obfuscation primitives are proposed to separate and aggregate the function, which are called fission and fusion respectively. A prototype of Khaos is implemented based on the LLVM compiler and evaluated on a large number of real-world programs including SPEC CPU 2006 & 2017, CoreUtils, JavaScript engines, etc. Experimental results show that Khaos outperforms existing code obfuscations and can significantly reduce the accuracy rates of five state-of-the-art binary diffing techniques (less than 19%) with lower runtime overhead (less than 7%).
Introduction
Binary diffing is a critical technique for identifying differences between program versions, often used for vulnerability discovery or patch analysis. However, modern diffing tools have evolved to use sophisticated semantic features and machine learning, treating the function as the atomic unit of analysis. Traditional obfuscation techniques, such as instruction substitution or control flow flattening, are primarily intra-procedural and fail to disrupt these function-level features. Khaos addresses this by breaking the stability of function boundaries through inter-procedural transformations.
Methodology: Fission and Fusion
Khaos is built on the LLVM compiler framework and introduces two core primitives: fission and fusion.
Fission involves splitting a single function into multiple sub-functions. This is achieved through dominator tree analysis to identify extractable regions while employing a "data-flow reduction" mechanism to minimize inter-procedural overhead.
Fusion aggregates multiple functions into a single entity. This requires managing complex call sites through parameter list compression and a tagged pointer mechanism to ensure correct runtime execution logic, even when functions are called indirectly.
A key innovation is Deep Fusion, which identifies "innocuous" basic blocks—segments that do not alter global memory state—and interleaves them across the merged functions to further blur the lines for analysis tools.
Evaluation and Results
The researchers evaluated Khaos against five state-of-the-art binary diffing tools (including BinDiff and Asm2Vec) using three datasets: SPEC CPU benchmarks (T-I), CoreUtils (T-II), and real-world IoT software with known vulnerabilities (T-III), such as JerryScript and OpenSSL.
- Diffing Accuracy: Khaos reduced the matching accuracy of all tested diffing tools to below 19%.
- Vulnerability Hiding: In tests on real-world vulnerabilities, the "escape ratio" was remarkably high. For instance, using Asm2Vec, over 80% of vulnerable functions were not found within the top-50 candidates after Khaos was applied.
- Performance: The geometric mean of runtime overhead was approximately 5-6%, staying well under the 7% threshold required for practical use in resource-constrained embedded devices.
Discussion and Conclusion
Khaos demonstrates that inter-procedural obfuscation is a powerful defense against modern binary analysis. By leveraging compiler optimizations to restructure code across function boundaries, it effectively blinds tools that rely on stable function-level features. While challenges remain in handling C++ exceptions and static pointer management, Khaos provides a lightweight and highly effective solution for protecting software in the IoT era. Future work will explore defenses against advanced data-flow analysis that might attempt to untangle these inter-procedural dependencies.
Summary updated