Functionality Preserving Adversarial Attacks on Software

28 concepts25 papers1 proposal

In computer vision, generating an adversarial example is conceptually straightforward: add a tiny, bounded vector of continuous pixel noise to an image. The resulting image looks identical to a human observer but completely misleads a deep neural network.

When we shift from continuous domains like images to discrete domains like computer software, this paradigm breaks down entirely. As Ihai Rosenberg et al. outline in their comprehensive survey, Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain, arbitrary bit-flipping or gradient-based shifts cannot be directly applied to software. A single random byte modification in a compiled binary can corrupt the file format or cause a segmentation fault. Similarly, a minor token change in source code can violate compiler syntax or fundamentally alter the program's logic.

To successfully attack machine learning models that analyze code or detect malware, researchers must operate under a strict semantic integrity constraint. The adversarial example must remain a fully functional program that executes its original logic perfectly, while simultaneously altering its feature representation enough to evade detection.


The Core Mechanism: Exploiting Non-Functional Regions

How do you change a file's statistical footprint without changing what it does? The core insight shared across this research domain is that software representations—whether compiled binaries or high-level source code—contain structural "blind spots" or non-functional regions that machine learning models over-rely on for classification.

1. Binary-Level Manipulation (Executables)

For compiled binaries (such as Windows PE files or Android APKs), models like MalConv often process raw byte sequences or structural metadata. Attackers exploit this by injecting adversarial perturbations exclusively into regions that do not affect the execution path:

2. Source Code-Level Manipulation

When attacking Programming Language (PL) models (like CodeBERT or CodeT5 used for vulnerability detection or code summarization), the constraint shifts from file format compliance to syntactic fluency and compilability.

  • Dead Code Insertion: In DIP: Dead code Insertion based Black-box Attack for Programming Language Model, CheolWon Na et al. inject non-functional control statements (e.g., if (false) { ... }) that are ignored during execution but force the model to process new, confusing token distributions.
  • Discrete Variable Renaming: Noam Yefet et al. introduced the DAMP framework in Adversarial Examples for Models of Code, which uses gradient-based optimization to select and rename variable identifiers. Because variable names carry heavy semantic weight for deep learning models but mean nothing to a compiler, this acts as a highly effective, semantics-preserving perturbation.
  • Fluency Preservation: Akshita Jha argues in CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models that code perturbations must maintain strict language-specific constraints (like operator counts and identifier types). This ensures the generated code remains fluent and natural to human reviewers, preventing the attack from being flagged by simple heuristic filters.

Bridging the Optimization Gap

Because software modifications are discrete, attackers cannot simply compute the gradient of the model's loss function with respect to the input and take a small step. Flipping a single bit from 0 to 1 is a non-differentiable jump. To overcome this, researchers have developed specialized optimization strategies that bridge the gap between continuous gradients and discrete, functionality-preserving constraints.

+------------------------+
| Original Program /     |
| Malicious Binary       |
+-----------+------------+
            |
            v
+------------------------+
| Discrete Transformation| <--- Constrained by Semantic Integrity
| Space Selection        |      (Slack space, Dead code, Renaming)
+-----------+------------+
            |
            v
+------------------------+
| Optimization Engine    | <--- Zeroth-Order (ZEXE), Genetic (GAMMA),
| (Search & Evaluate)    |      or Reinforcement Learning (RLAttackNet)
+-----------+------------+
            |
            v
+------------------------+
| Evaded Model /         |
| Functional Adversarial |
+------------------------+

Joint Optimization and Obfuscation

Instead of treating the attack as a random search, Shashank Srikant et al. framed the problem mathematically in Generating Adversarial Computer Programs using Optimized Obfuscations. They treated "where to change" and "what to change" as a joint optimization problem, mapping semantic equivalence rules directly into the loss function to guide the generation of adversarial programs.

Zeroth-Order and Genetic Algorithms

When target models are black boxes or gradients are completely inaccessible due to the discrete nature of the file format, alternative optimization frameworks are required:

Reinforcement Learning and Tree Searches

Other approaches treat the generation of functionality-preserving perturbations as a sequential decision-making problem. Yong Fang et al. developed RLAttackNet in DeepDetectNet vs RLAttackNet: An adversarial method to improve deep learning-based static malware detection model, training a reinforcement learning agent to autonomously discover which sequence of format-preserving modifications will successfully slip past a deep learning detector. Similarly, Ping He et al. organized potential modifications into a structured "Perturbation Selection Tree" in Efficient Query-Based Attack against ML-Based Android Malware Detection under Zero Knowledge Setting, allowing efficient query-based navigation through the discrete search space.


Why It Works: The Illusion of Comprehension

The ultimate reason these attacks are so devastatingly effective is that current machine learning models for software analysis do not truly understand program semantics. Instead, they rely on statistical correlations—such as token frequencies, byte distributions, or local graph structures.

By operating strictly within semantic integrity constraints, frameworks like RAMEN, DAMP, and ZEXE leave the core execution logic untouched while completely scrambling the surface-level statistical patterns. The model looks at the modified program, sees a distribution of tokens or bytes that closely resembles a harmless utility program or a clean code snippet, and misclassifies it—while the underlying malicious payload or vulnerability remains fully intact and ready to execute.

Key Foundational Work

Black-Box Attacks against RNN based Malware Detection Algorithms (2017) — contributed 1 concept elements to this direction.
Black-Box Obfuscation for d-CNFs (2013) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_semantic_integrity_constraint_37e9009b,
  title = {semantic integrity constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-integrity-constraint/rdr_37e9009bb3a14d74921c187ddefdaed3},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_semantic_integrity_constraint_37e9009b_v1,
  title = {semantic integrity constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/semantic-integrity-constraint/rdr_37e9009bb3a14d74921c187ddefdaed3/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake