Structural Code Representation via Abstract Syntax Trees

24 concepts23 papers1 proposal

When humans read source code, they see a sequence of text characters. When compilers read source code, they see a highly structured, mathematical graph called an Abstract Syntax Tree (AST).

For a long time, machine learning models treated code like natural language—as a flat sequence of tokens. However, this text-first assumption forces models to waste capacity learning basic syntax rules and filters out the rich, hierarchical logic inherent to programming languages. A major research direction focuses on extracting and modeling features directly from ASTs. By shifting the representation from flat text to structured trees, models can ignore superficial variations and focus entirely on the underlying algorithmic logic.

Beyond the Flat Text Assumption

Why does treating code as plain text fall short? In natural language, the order of words dictates meaning, but the structure is fluid. In programming, code is governed by a strict, nested grammar. A flat sequence model gets easily distracted by "surface noise"—such as variable renaming, formatting choices, or language-specific boilerplate syntax.

An AST solves this by mapping a program's execution logic into a hierarchy of root, internal, and leaf nodes. For example, a loop statement becomes a parent node, while its conditions and inner blocks become its children.

As noted in a comprehensive survey by Xiaotao Song et al. in A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques, integrating ASTs into code analysis models captures formal dependencies and nested program flows far better than token sequences. This structural grounding ensures that the model evaluates what the code does, rather than just how it looks.

Achieving Invariance to Surface-Level Noise

The primary reason AST-based feature extraction works so well is structural invariance. If you change a variable name from counter to x, or if you translate an algorithm from C++ to Java, the raw text changes completely. However, the shape of the underlying AST remains remarkably stable.

This invariance unlocks several capabilities across different domains:

Mechanics of AST Feature Extraction

To feed a tree structure into machine learning models, researchers utilize a few distinct structural paradigms:

Linearization vs. Tree-Based Networks

While some architectures process trees directly via tree-based convolutions, others flatten the tree using deterministic traversals (like depth-first search) to create a linearized AST. Johannes Villmow et al. showed in ConTest: A Unit Test Completion Benchmark featuring Context that these linearized sequences explicitly preserve the hierarchical nesting of code, providing a massive structural advantage for code generation models over raw tokenized source text. When generating code from scratch, Yabing Zhu et al. in GANCoder: An Automatic Natural Language-to-Programming Language Translation Approach Based on GAN used generative adversarial networks to output ASTs directly, ensuring that the synthesized programs adhered strictly to valid hierarchical structures rather than risking syntax corruption.

Node Filtering and Enrichment

Not every node in an AST is equally informative. Researchers often filter or enrich the tree to optimize downstream performance. For instance, Hung Phan et al. developed a method in Evaluating and Optimizing the Effectiveness of Neural Machine Translation in Supporting Code Retrieval Models: A Study on the CAT Benchmark that isolates a specific subset of non-terminal nodes within the AST. This compressed representation captures the core structural elements of the program, making it easier for translation models to map natural language queries to code.

Conversely, Chongyu Zhang et al. chose an enrichment strategy in BERT-Based Code Learning for Exception Localization and Type Prediction, parsing the AST to extract structural features while systematically substituting variable names to retain high-level class information, which significantly improved exception localization.

Programmatic Manipulation and Static Analysis

Because an AST is a formal, deterministic data structure, it bridges the gap between statistical machine learning and precise static analysis. Developers can programmatically query or instrument code with 100% accuracy, avoiding the guesswork of text-based regex patterns.

Visitor Patterns and Instrumentation

In frameworks like the one proposed by Chao Peng et al. in SIF: A Framework for Solidity Contract Instrumentation and Analysis, developers interact with the AST via "visitor functions." These functions trigger automatically at specific node types (e.g., whenever the parser encounters a smart contract function call), allowing for seamless structural transformations.

For dynamic analysis, Shreya Shankar et al. demonstrated in Bolt-on, Compact, and Rapid Program Slicing for Notebooks [Scalable Data Science] that applying instrumentation at the AST level offers far greater portability and flexibility than low-level bytecode instrumentation. This allows tools to selectively track variables and slice code blocks directly within interactive computational notebooks.

Broad Downstream Applications

The utility of AST feature extraction spans across software engineering, security, and performance modeling:

Key Foundational Work

Syntactic and Functional Variability of a Million Code Submissions in a Machine Learning MOOC (2013) — contributed 1 concept elements to this direction.
Glaucus: Predicting Computing-Intensive Program's Performance for Cloud Customers (2013) — contributed 1 concept elements to this direction.
A Validation of Object-Oriented Design Metrics as Quality Indicators (1996) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_structural_code_representation_ast_based_51ffc2bc,
  title = {structural code representation (AST-based feature extraction)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-code-representation-ast-based-feature-extraction/rdr_51ffc2bcdbaa45389fe732e6e5ef3e83},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_structural_code_representation_ast_based_51ffc2bc_v1,
  title = {structural code representation (AST-based feature extraction)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/structural-code-representation-ast-based-feature-extraction/rdr_51ffc2bcdbaa45389fe732e6e5ef3e83/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake