Identifier Abstraction and Code Normalization for Machine Learning

134 concepts101 papers1 proposal

When humans read source code, they rely heavily on lexical hints. A variable named num_elements or a function named sort_array instantly communicates intent. However, for neural models trained on source code, this reliance on human-readable names is a double-edged sword. Models frequently overfit to these superficial naming conventions, treating two structurally identical algorithms as completely different if their variable names change, or conversely, misinterpreting a broken algorithm because its variables are deceptively well-named.

This vulnerability is particularly evident in tasks like vulnerability detection and code clone detection. As shown by Zhen Li 0027 et al. in their work on VulDeePecker, deep learning models can easily get distracted by project-specific naming styles, failing to recognize the underlying structural patterns of a security flaw. To build models that truly understand program logic, researchers have turned to identifier abstraction and structural code normalization—techniques that strip away superficial stylistic variations to expose the invariant mathematical and logical structure of code.

The Mechanics of Identifier Anonymization

The most direct way to force a model to look past surface-level syntax is to eliminate descriptive names entirely. This process, often referred to as identifier anonymization or "lexical erosion," replaces user-defined variable, function, and class names with generic, ordered placeholders (e.g., var1, var2, func1).

Consider a simple function that computes a factorial. Whether the loop counter is named i, counter, or idx, the underlying data flow graph remains identical. By mapping these diverse strings to a standardized pool of tokens, we achieve several critical benefits:

  1. Vocabulary Reduction: Instead of dealing with an infinite vocabulary of creative developer names, the model operates on a small, finite set of tokens. As noted in the impact of lexical and grammatical processing on generating code from natural language, transforming an infinite vocabulary into a finite set can be more effective for model performance than scaling up the semantic depth of the encoder.
  2. Structural Isolation: In Empirical Study of Transformers for Source Code, Nadezhda Chirkova demonstrates that when identifiers are completely anonymized, Transformers are forced to learn exclusively from the structural arrangement and formal grammar of the code, leading to more robust generalization on unseen codebases.

To maintain the relational logic of the program without the literal strings, architectures like the one proposed by Elizabeth A Dinella et al. in Hoppity separate the structural topology of the code from its lexical identifiers, using pointer mechanisms to reference local context rather than memorizing specific strings.

Canonicalization and Structural Normalization

Identifiers are not the only source of superficial noise; control flow and formatting also introduce arbitrary variations. Two programs can be semantically identical but look completely different due to formatting choices, loop structures, or language-specific idioms. Code canonicalization solves this by rewriting diverse implementation styles into a single uniform representation.

For instance, in code translation, models often struggle with one-to-many mappings where a single high-level concept can be expressed via multiple low-level constructs. To address this, Shuoming Zhang et al. pre-processed training data to force a single translation style (such as converting all switch-case statements into if-else structures), preventing the model from being confused by syntactic variance. Similarly, Aniketh Malyala et al. demonstrated that reducing complex language-specific loop structures into basic universal constructs like while-loops significantly improves translation accuracy.

Beyond manual rewriting rules, researchers leverage compilers and decompilers as ultimate normalization engines. Compilers are designed to strip away syntactic sugar and optimize code into a standardized intermediate representation (IR). Pingchuan Ma 0004 et al. utilized compiler optimization passes to reduce syntactic noise, transforming varied coding styles into standardized intermediate structures. On the flip side, when dealing with compiled binaries, Qibin Chen et al. used decompilers as a preprocessing step to normalize low-level compiler optimizations back into a consistent high-level representation, allowing models to maintain high accuracy across heavily optimized binaries.

Why Normalization Enhances Generalization

Why does stripping away information make neural models perform better? The answer lies in the bias-variance tradeoff of representation learning.

When a model trains on raw source code, it learns a shortcut: it associates specific variable names or formatting patterns with specific behaviors. If a training set contains vulnerabilities written by a developer who always names their buffer pointer buf, the model learns to flag buf rather than analyzing the missing bounds check. By applying identifier abstraction—as done in the vulnerability detection framework by Di Cao in RealVul—the model is denied this shortcut. It is forced to track the actual data flow and control dependencies.

This abstraction is equally powerful in low-level binary analysis. In cross-platform binary similarity detection, memory addresses and constants change completely depending on the target architecture or compilation offset. By abstracting constants and memory addresses into generic tokens, models like the one described in Neural Machine Translation Inspired Binary Code Similarity Comparison beyond Function Pairs by Fei Zuo et al. mitigate the out-of-vocabulary problem and focus purely on the structural logic of the basic blocks.

Ultimately, normalization transforms the problem of code intelligence from a complex natural-language-and-syntax hybrid task into a pure structural reasoning task, unlocking higher data efficiency and truer semantic understanding.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_identifier_abstraction_e14b494d,
  title = {identifier abstraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identifier-abstraction/rdr_e14b494d3a994fd881cbc1332d6bc738},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_identifier_abstraction_e14b494d_v1,
  title = {identifier abstraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/identifier-abstraction/rdr_e14b494d3a994fd881cbc1332d6bc738/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake