Abstract: Machine learning and deep learning in particular has been recently used to successfully address many tasks in the domain of code such as finding and fixing bugs, code completion, decompilation, type inference and many others. However, the issue of adversarial robustness of models for code has gone largely unnoticed. In this work, we explore this issue by: (i) instantiating adversarial attacks for code (a domain with discrete and highly structured inputs), (ii) showing that, similar to other domains, neural models for code are vulnerable to adversarial attacks, and (iii) combining existing and novel techniques to improve robustness while preserving high accuracy.
Introduction
Neural networks have become surprisingly effective at "understanding" source code, powering tools for variable type prediction, bug detection, and code completion. However, these models share a common vulnerability with computer vision models: they are remarkably fragile. Simple, semantics-preserving changes—such as renaming a variable from hex to color—can completely derail a model's prediction. This paper addresses this fragility by proposing a framework that makes models of code significantly more resilient to adversarial attacks.
The Challenge of Robustness in Code
In computer vision, adversarial attacks often involve subtle pixel changes. In code, attacks are discrete and structural. Modifications like renaming variables, adding "dead code," or swapping equivalent control structures (e.g., ternary operators for if-else blocks) are routine for developers but catastrophic for neural models.
The challenge is threefold:
- Structure: Programs are highly structured; local changes can have global effects.
- Disruption: Discrete changes are more disruptive than continuous pixel perturbations.
- Undecidability: Determining program equivalence is undecidable, complicating the definition of "safe" modifications.
Methods
The authors propose a multi-step pipeline to build a robust model, combining model abstention, adversarial training, and a novel representation refinement technique.
- Learning to Abstain: The model is allowed to "abstain" (output nothing) when uncertain, partitioning data into "easy/certain" and "hard/uncertain" buckets.
- Adversarial Training: The model is trained on "worst-case" versions of code. The system searches for small modifications that cause failure and teaches the model to handle them.
- Representation Refinement: Using Integer Linear Programming (ILP), the authors prune the program's graph representation to the minimal set of edges and nodes required for prediction. This reduces the "attack surface" by removing irrelevant code that could be used to trick the model.
- Specialized Ensembles: A sequence of models is trained, where each subsequent model specializes in the samples the previous models chose to abstain from.
Experimental Evaluation
The approach was evaluated on Type Prediction for JavaScript and TypeScript using a dataset of 3,000 top-starred GitHub projects, resulting in approximately 140,000 training samples. Ground truth labels were extracted using the TypeScript compiler.
Key Results
The framework was tested against several architectures (LSTM, DeepTyper, GNT, GGNN).
- Accuracy vs. Robustness: The best robust model (GNT architecture) achieved 87.7% accuracy and improved robustness from 52.1% to 67.0%.
- High-Reliability Subset: For approximately 29% of samples, the approach achieved 99.9% accuracy and robustness.
- Synergy of Techniques: Adversarial training alone only improved robustness by 0-7%. Significant gains required the combination of refinement and abstention.
Conclusion and Future Work
While effective, the approach has limitations, including the computational cost of training ensemble sequences and the task-specific nature of the current refinement process. Future work may involve making representation refinement end-to-end trainable, potentially eliminating the need for a separate ILP solver. This research highlights that while standard models are fragile, combining formal methods with deep learning can significantly enhance the reliability of AI for code.
Summary updated