Virtual Compiler Is All You Need For Assembly Code Search

Zeyu Gao, Hao Wang, Yuanda Wang, Chao Zhang

Abstract

Assembly code search is vital for reducing the burden on reverse engineers, allowing them to quickly identify specific functions using natural language within vast binary programs. Despite its significance, this critical task is impeded by the complexities involved in building high-quality datasets. This paper explores training a Large Language Model (LLM) to emulate a general compiler. By leveraging Ubuntu packages to compile a dataset of 20 billion tokens, we further continue pre-train CodeLlama as a Virtual Compiler (ViC), capable of compiling any source code to assembly code. This approach allows for “virtual” compilation across a wide range of programming languages without the need for a real compiler, preserving semantic equivalency and expanding the possibilities for assembly code dataset construction. Furthermore, we use ViC to construct a sufficiently large dataset for assembly code search. Employing this extensive dataset, we achieve a substantial improvement in assembly code search performance, with our model surpassing the leading baseline by 26%.

Introduction

Reverse engineering is a critical task in cybersecurity, often likened to finding a needle in a haystack. Security researchers analyzing binary files must identify specific functions, such as encryption routines or backdoors, within millions of lines of assembly code. Traditionally, this process is manual and tedious, relying on searching for specific strings or constants.

The ideal solution is Assembly Code Search, which allows researchers to use natural language queries (e.g., "AES encryption") to locate relevant assembly snippets. However, training effective models for this task requires massive datasets of paired natural language descriptions and assembly code. Creating these datasets is notoriously difficult—a phenomenon the authors call "compilation hell." Compiling source code at scale often fails due to missing dependencies or incompatible environments. This paper introduces a "Virtual Compiler" (ViC) to bypass these hurdles.

Methodology: The Virtual Compiler Approach

The core innovation of this work is training a Large Language Model (LLM) to emulate a compiler. Instead of relying on traditional build systems, the authors treat compilation as a sequence-to-sequence translation task.

The three-stage training pipeline: dataset construction, virtual compiler training, and assembly code encoder training.

The process involves three main stages:

  1. Code Dataset Construction: The researchers extracted 6,000+ C/C++ packages from Ubuntu repositories to create a ground-truth dataset of 15 million source-to-assembly pairs, totaling 20 billion tokens.
  2. Virtual Compiler Training: They fine-tuned CodeLlama-34B on this dataset. The resulting model, ViC, can translate source code into assembly while respecting various compiler flags (GCC vs. Clang) and optimization levels (O0 to O3).
  3. Assembly Code Encoder Training: Using ViC, the authors "virtually compiled" existing high-level datasets like CodeSearchNet (containing Python, Java, Go, etc.) into assembly. They then trained an Assembly Code Encoder using contrastive learning, aligning natural language queries with their corresponding assembly snippets in a shared vector space.

Datasets and Evaluation

The study utilized several key datasets to validate the approach:

  • ViC Training Set: 15 million pairs derived from Ubuntu packages.
  • HybridAssembly: A large-scale search training set combining the C-based CCSD dataset and the multi-language CodeSearchNet, all augmented with virtual assembly generated by ViC.
  • Real-World Evaluation: A manually curated set of 257 queries and function pairs from Linux, Mac, and Windows binaries to test cross-platform transferability.

Key Results and Performance

The model trained on ViC-generated data, dubbed HybridAssembly, demonstrated superior performance over existing state-of-the-art (SOTA) embedding models.

Comparison of HybridAssembly against leading embedding models on assembly code search metrics.

As shown in the table above, HybridAssembly achieved a Recall@1 of 0.424, significantly outperforming models like OpenAI’s text-embedding-3-large (0.344) and Voyage AI’s voyage-code-2 (0.331). This represents a substantial 26% improvement over the leading baseline in identifying the correct assembly snippet on the first attempt.

Insights and Discussion

A surprising finding was ViC's ability to generalize. Although trained primarily on C/C++, it successfully "compiled" languages like Go and Python. This suggests the LLM learned the fundamental mapping between high-level logic (loops, conditionals) and low-level instructions.

While the virtual assembly might not be bit-for-bit identical to real compiler output—particularly regarding specific memory addresses—it remains semantically equivalent. For code search, preserving the functional meaning and control flow is more critical than literal accuracy. This approach effectively bypasses the environmental dependencies that make traditional large-scale compilation impossible.

Conclusion

The Virtual Compiler (ViC) offers a scalable solution for generating high-quality assembly code datasets. By leveraging synthetic data, the researchers achieved a new state-of-the-art in assembly code search, providing a powerful tool for reverse engineers to navigate complex binary programs using natural language.

Summary updated

Abstract

Assembly code search is vital for reducing the burden on reverse engineers, allowing them to quickly identify specific functions using natural language within vast binary programs.Despite its significance, this critical task is impeded by the complexities involved in building high-quality datasets. This paper explores training a Large Language Model (LLM) to emulate a general compiler. By leveraging Ubuntu packages to compile a dataset of 20 billion tokens, we further continue pre-train CodeLlama as a Virtual Compiler (ViC), capable of compiling any source code to assembly code. This approach allows for “virtual” compilation across a wide range of programming languages without the need for a real compiler, preserving semantic equivalency and expanding the possibilities for assembly code dataset construction. Furthermore, we use ViC to construct a sufficiently large dataset for assembly code search. Employing this extensive dataset, we achieve a substantial improvement in assembly code search performance, with our model surpassing the leading baseline by 26%.
BibTeX
@inproceedings{gao2024virtual,
  author = {Zeyu Gao and Hao Wang and Yuanda Wang and Chao Zhang},
  title = {{Virtual Compiler Is All You Need For Assembly Code Search}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=s1ioXw3lvC},
}
Figure