In software engineering and security, we frequently need to answer a fundamental question: Have we seen this piece of code before? Whether trying to eliminate duplicate data in machine learning datasets, identifying plagiarized assignments, tracking down vulnerable third-party libraries, or uncovering cloned smart contracts, the core challenge remains the same.
Traditional cryptographic hashing algorithms like SHA-256 are entirely unsuited for this task. They are designed to be fragile; changing a single character or variable name alters the output hash completely. In the real world, code changes constantly due to refactoring, variable renaming, compiler optimizations, or minor structural adjustments.
To bridge this gap, researchers have developed fuzzy code matching. Instead of treating code as an exact string of characters, fuzzy matching extracts robust invariants—syntactic, structural, or semantic—allowing systems to recognize functionally identical code even when its surface appearance has been heavily modified.
Syntactic and Token-Based Invariants
The simplest way to move past exact string matching is to look at the statistical distribution of tokens or characters. By stripping away cosmetic variations like comments and formatting, we can isolate the core vocabulary of a code snippet.
In the context of dataset deduplication, Allamanis demonstrated in The adverse effects of code duplication in machine learning models of code that extracting identifiers and literals and computing their Jaccard similarity is highly effective at catching near-duplicate files. Because developers often copy-paste code while only changing minor variable names, the underlying token overlap remains remarkably high.
Taking this a step further, Barrón-Cedeño et al. showed in DeSoCoRe: Detecting Source Code Re-Use across Programming Languages that character n-gram frequency vectors can detect code reuse even across different programming languages (such as Java to C++). By calculating the cosine similarity of these normalized vectors, the system bypasses language-specific syntax to capture the shared statistical footprint of the underlying logic.
Source Code A: int count = 0; while(count < 10) { count++; }
Source Code B: int i = 0; while(i < 10) { i++; }
Tokenized A: [INT, ID, ASSIGN, NUM, WHILE, LPAREN, ID, LT, NUM, RPAREN, ...]
Tokenized B: [INT, ID, ASSIGN, NUM, WHILE, LPAREN, ID, LT, NUM, RPAREN, ...]
▲ Both map to the exact same token sequence!
Structural and Control-Flow Alignment
Token distributions can be fooled if a developer significantly reorders statements or if a compiler aggressively optimizes the bytecode. To counter this, advanced fuzzy matching looks at the structural skeleton of the program: its Abstract Syntax Tree (AST) or Control Flow Graph (CFG).
When analyzing decentralized exchanges for malicious activity, Huynh et al. utilized an AST-Jaccard score in Serial Scammers and Attack of the Clones: How Scammers Coordinate Multiple Rug Pulls on Decentralized Exchanges to quantify the structural resemblance between smart contracts. Because the AST represents the nested logical hierarchy of the code, it remains stable even if variable names are completely randomized.
When source code is unavailable—such as when auditing compiled Android binaries or web scripts—researchers look at execution profiles or opcode sequences:
- Fuzzy Hashing on Opcodes: In ATVHunter: Reliable Version Detection of Third-Party Libraries for Vulnerability Identification in Android Applications, Zhan et al. applied fuzzy hashing directly to opcode sequences alongside structural control-flow analysis. This ensures that minor variations introduced by different compiler optimization flags do not obscure the library's identity.
- Structural Signatures at Runtime: Similarly, Ali et al. developed a runtime structural fingerprinting mechanism in Unbundle-Rewrite-Rebundle: Runtime Detection and Rewriting of Privacy-Harming Code in JavaScript Bundles to identify specific versions of bundled JavaScript libraries, matching them against a database of known tracking scripts using a defined similarity threshold.
Continuous Vector Embeddings and Extrapolation
The most powerful evolution of fuzzy code matching moves away from discrete graph or token comparisons entirely, mapping code into a continuous, high-dimensional vector space. In this space, geometric distance (such as Euclidean or Cosine distance) directly corresponds to structural or functional similarity.
In their seminal work Vulnerability Extrapolation: Assisted Discovery of Vulnerabilities Using Machine Learning, Yamaguchi et al. introduced the concept of "vulnerability extrapolation." By embedding source code into a vector space and using Principal Component Analysis (PCA) to capture latent API usage patterns, they showed that a known vulnerable function could be used as a "seed." A simple nearest-neighbor search in this reduced vector space reveals other functions sharing the same structural flaws, turning a needle-in-a-haystack security audit into a targeted recommendation task.
Known Vulnerable Seed (Vector V)
│
▼
┌────────────────────────────────────────────────────────┐
│ Vector Space │
│ │
│ Func_A Func_B │
│ ○ ○ │
│ │
│ Seed (V) │
│ ● │
│ \ Nearest Neighbor │
│ \ │
│ ○ Zero-Day Candidate! │
│ Func_C │
└────────────────────────────────────────────────________┘
This embedding strategy has been widely adopted for smart contract auditing. In SmartEmbed: A Tool for Clone and Bug Detection in Smart Contracts through Structural Code Embedding, Jayasundara et al. framed bug detection as a similarity search problem against a curated database of known buggy code fragments. As further detailed by Jiang et al. in Checking Smart Contracts With Structural Code Embedding, this numerical abstraction allows tools to rapidly flag complex, refactored code clones that completely bypass traditional rule-based static analysis.
The Challenge of Near-Identical Code
While mapping code to nearby vector space coordinates works well for finding broad structural clones, it introduces a unique challenge in security: benign and vulnerable code are often 99% identical. A secure function and a vulnerable function might differ by only a single line—such as a missing bounds check.
To address this limitation, Le et al. proposed a specialized deep learning architecture in Maximal Divergence Sequential Autoencoder for Binary Software Vulnerability Detection. Instead of letting the model naturally group similar code structures together, their autoencoder explicitly forces the internal representations of "vulnerable" and "clean" code to be as distant as possible in the latent space. By artificially stretching the space between safe and unsafe variants, the model overcomes the blind spots of standard discriminative models, allowing for highly precise boundary detection.
Aggregating Local Similarities for Global Context
When dealing with large software packages or compiled binaries, matching a single function or snippet is rarely enough. Code similarity frameworks must aggregate local, function-level matches to make global decisions about software identity.
For instance, in LibDB: An Effective and Efficient Framework for Detecting Third-Party Libraries in Binaries, Tang et al. combined deep-learning-based function embeddings with graph-based filtering. To pinpoint the exact version of a library embedded inside a stripped binary, their framework aggregates the individual similarity scores of all matched functions. This holistic ranking strategy allows the system to detect subtle semantic shifts across library versions, successfully disambiguating closely related releases based on the cumulative weight of minor code changes.
Key Foundational Work
Vulnerability Extrapolation: Assisted Discovery of Vulnerabilities Using Machine Learning (2011) — contributed 2 concept elements to this direction.
DeSoCoRe: Detecting Source Code Re-Use across Programming Languages (2012) — contributed 1 concept elements to this direction.
Selecting peers for execution comparison (2011) — contributed 1 concept elements to this direction.