Identifier Dependency in Code Language Models

46 concepts34 papers

In programming languages, developers enjoy symbolic freedom. The functional logic of a program is entirely decoupled from the arbitrary names assigned to its variables, functions, and classes. A compiler treats a variable named total_sum exactly the same as one named q3_x.

However, as neural network models have become the dominant tools for code analysis, search, and generation, researchers have uncovered a striking paradox: these models derive a massive portion of their performance not from understanding pure program logic, but from reading human-authored identifier names. When these descriptive labels are removed, randomized, or altered, the models' apparent comprehension frequently collapses. This phenomenon is known as semantic identifier dependency.

The Statistical Shortcut: Natural vs. Formal Channels

Source code contains two concurrent streams of information: the formal channel (the strict execution logic, control flow, and syntax rules) and the natural channel (human-readable identifiers and comments). Because deep learning models are typically trained on massive corpora of open-source code, they are exposed to strong statistical correlations between descriptive names and specific algorithmic behaviors.

Instead of learning the computationally demanding task of tracing data flows or simulating execution paths, models take a statistical shortcut. They treat identifier names as primary semantic anchors. Early structural models like code2vec: Learning Distributed Representations of Code by Uri Alon et al. and early generation systems like CODE-NN by Srinivasan Iyer et al. demonstrated that when code is obfuscated or uses non-descriptive identifiers, predictive performance degrades significantly. The models lose the textual scaffolding they rely on to interpret syntactic structures.

As Zhilong Wang et al. demonstrated in How Does Naming Affect Language Models on Code Analysis Tasks?, stripping or anonymizing identifiers can cause a model's performance on tasks like code search to plummet by approximately 75%. This reveals that code language models often prioritize literal, surface-level features over underlying logical structure and control flow.

Exposing Fragility via Adversarial Renaming

The depth of this dependency becomes obvious when models are subjected to adversarial perturbations. Because models over-rely on the natural channel, making minor, non-functional changes to human-readable elements can completely disrupt their predictions.

In Adversarial Examples for Models of Code, Noam Yefet et al. introduced a white-box attack (DAMP) that renames variables using gradient-based optimization. By changing just a few identifier names without altering program behavior, they successfully tricked models into making specific, incorrect predictions up to 89% of the time. Similarly, the black-box attack framework CodeAttack, developed by Akshita Jha in CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models, showed that state-of-the-art models like CodeT5 and CodeBERT are highly vulnerable to human-imperceptible modifications of variable names and operators.

# Original Code: Model easily understands intent
def calculate_area(radius):
    return 3.14 * (radius ** 2)

# Adversarially Renamed Code: Model performance plummets
def download_file(payload):
    return 3.14 * (payload ** 2)

This fragility is not limited to variable names; it extends to function signatures. Nuo Chen et al. found in Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation that if a function signature is modified to contradict the internal code behavior, the model will prioritize the misleading signature over the actual operational logic.

This heuristic bias can even lead to inverse scaling in large language models. Antonio Valerio Miceli Barone et al. showed in The Larger they are, the Harder they Fail: Language Models do not Recognize Identifier Swaps in Python that when common built-in functions (like len and print) are explicitly redefined or swapped within a local scope, larger models fail more confidently than smaller ones. Their strong pre-trained lexical correlations override their ability to track scope and binding rules.

Quantifying the Semantic Gap

When models are forced to operate in environments stripped of natural language hints, their lack of deep structural understanding becomes apparent. This semantic gap has been quantified across several distinct dimensions:

Breaking the Dependency: Grounding Models in Logic

To move past these superficial heuristics, current research focuses on forcing models to learn the underlying algorithmic logic of code independent of its naming conventions.

Obfuscation Grounding

One direct solution is to alter the pre-training distribution so that models cannot rely on lexical shortcuts. In ObscuraCoder: Powering Efficient Code LM Pre-Training Via Obfuscation Grounding, Indraneil Paul et al. introduced a pre-training strategy where descriptive variable and function names are systematically replaced with generic placeholders. This forces the model to learn algorithmic logic from the ground up, yielding higher data efficiency and superior performance on complex reasoning tasks.

Contrastive and Template-Based Learning

Other approaches use specialized training objectives to enforce identifier invariance. The authors of ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning applied contrastive learning with data augmentations to explicitly teach models that a program's semantic meaning remains identical when identifiers are renamed.

To achieve true compositional generalization, Hojae Han et al. proposed CTBERT in Towards Compositional Generalization in Code Search. CTBERT automatically mines structural templates from Abstract Syntax Trees (ASTs), training the model to recognize structural idioms rather than relying on lexical pattern matching.

Code Erosion

For downstream tasks like code summarization, frameworks like VECOS, developed by Fan Yan et al. in Towards Generating Summaries for Lexically Confusing Code through Code Erosion, intentionally "erode" code into a standardized, identifier-neutral format. By using variational inference to align these eroded representations with their original functional meanings, the system ensures that summaries are generated based on structural logic rather than misleading or non-standard naming conventions.

Key Foundational Work

code2vec: Learning Distributed Representations of Code (2018) — contributed 2 concept elements to this direction.
Summarizing Source Code using a Neural Attention Model (2016) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_identifier_dependency_74322565,
  title = {semantic identifier dependency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-identifier-dependency/rdr_743225659f65473aa70e7d3204ba1fe7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_semantic_identifier_dependency_74322565_v1,
  title = {semantic identifier dependency},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-identifier-dependency/rdr_743225659f65473aa70e7d3204ba1fe7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake