Deep learning vulnerability detectors trained on synthetic test suites look excellent in the lab and fail in real software. Models reported to reach up to 95% accuracy on template-generated suites such as Juliet lose more than half their performance on de-duplicated Chromium and Debian code. The cause is that they memorize surface features of the synthetic data, such as variable names and boilerplate, instead of learning what makes code unsafe (Chakraborty et al.). Sentiment analysis solved a similar problem a decade ago. Word embeddings place "good" and "bad" close together because the two words appear in the same contexts. Concatenating each word's embedding with a small vector of scores from several hand-built sentiment dictionaries (a lexicon) fixed this blind spot and made models much less data-hungry (lingCNN, IWV). Code has its own lexicons: the rule sets of static analyzers, tools that flag risky patterns such as an unchecked copy into a fixed buffer. A rule like that fires on a Juliet test case for the same reason it fires on Chromium. In this setting static analyzers have been used only as weak baselines or as noisy label generators, and never as an input channel.
We propose to attach multi-analyzer rule-hit vectors to every statement node of a code graph, and to spread those scores along data-flow edges. Data-flow edges link the place where a variable gets its value to the places where that value is used. Spreading scores this way means a statement that matches no rule but receives tainted data still inherits a signal. The claim is that this channel shifts much less between synthetic and real code than token identity does. A detector trained only on synthetic labels that is pushed to rely on this channel should therefore close a measurable part of the synthetic-to-real gap. The proposal also tests this mechanism directly instead of assuming it.
Core Claim
The three research directions play different roles here. The lexicon-aggregation work in sentiment analysis supplies the mechanism: orthogonal, hand-built feature channels concatenated at the token level, with word-level injection outperforming sentence-level injection (lingCNN). The structural-representation work fixes the main known weakness of lexicons. Lexicon features are unigram-level and default to zero for terms not in the dictionary (IWV). Code graphs let a rule hit at a buffer allocation reach the sink that overflows it many lines later, which flat token sequences struggle to connect (Xu Duan et al.). The synthetic-to-real gap is the setting and the test. The falsifiable statement is the following. For a detector trained on Juliet and evaluated on REVEAL, statement-level, graph-propagated rule features reduce the transfer gap more than three alternatives: the same rule features fused at the classifier head, identifier normalization, and unsupervised feature alignment. They should also make the detector less sensitive to identifier renaming on real code.
Relation to Prior Work
Several papers motivate the gap. Chakraborty et al. diagnose the synthetic-to-real collapse and propose REVEAL, which runs a gated graph neural network over code property graphs. A code property graph merges the syntax tree, control flow, and data flow of a program into one graph. REVEAL uses no external knowledge, so it is a backbone we reuse rather than a competitor in kind. FGVulDet supplies the encoder: an edge-aware graph network with separate parameters for syntax, control-flow, and data-flow edges. Its slicing-guided augmentation addresses data scarcity, not domain shift. BUGFARM shows that subtle synthetic bugs placed in low-attention statements raise false-negative rates by up to 40.53%. It is a stress test only, and we use it as one.
Other work constrains the design. Dong et al. show that global feature alignment between code projects causes negative transfer. Their method, MNCRI, combines mutual-nearest-neighbor contrastive alignment with instance re-weighting, and was evaluated only between real projects. This warns us against forcing invariance on the learned representation. Our invariance comes from an input channel that is designed to be stable, not from alignment losses. Li et al. show in LiDAR transfer that unnaturally clean synthetic data gives a model an easy shortcut. They fix this by masking synthetic inputs. We borrow the idea in a simple form: random dropout of the identifier and token channels on synthetic training graphs, so the model cannot rely only on Juliet's surface regularities.
D2A is the closest use of static rules together with real-world data. It filters Infer reports by checking which ones disappear after bug-fix commits, and trains simple classifiers on report metadata, reaching a 70–80% false-positive reduction rate. D2A treats analyzer output as the thing to be classified. We treat it as a feature of the code. D2A also leaves graph models on its data as future work. MoCQ closes a loop between an LLM and symbolic graph queries, reaching 0.79 recall and 0.39 precision. Its output is a set of static queries, not a learned detector, and it does not study transfer. Its Joern-based queries are, however, a natural third rule source for our lexicon. GraphCodeBERT is a code transformer pretrained with data-flow edges as attention constraints. It gives us a second backbone and a ready data-flow graph to which rule features can be attached.
Research Program
We first build the code lexicon. We run three analyzer families over every function in the source and target data. The first is a pattern-level C checker in the style of Flawfinder. The second is the Infer analyzer used by D2A. The third is a small library of Joern graph queries patterned on MoCQ's core query subset. Rule hits are grouped into roughly two dozen families, such as unbounded copy, unchecked allocation, integer narrowing, and use after release, separately for each analyzer. This gives each statement node a short vector of hit indicators and confidences. That mirrors IWV stacking six sentiment dictionaries into a 6-dimensional channel, and it keeps the disagreement between analyzers visible instead of collapsing it to a single vote.
Next we spread the scores along the graph. If $r_v$ is the raw rule vector of node $v$, the propagated vector is
$$\tilde r_v = r_v + \sum_{k=1}^{K} \gamma^k \sum_{u \in N_k(v)} r_u ,$$
where $N_k(v)$ is the set of nodes that reach $v$ through exactly $k$ data-flow define-use edges, $\gamma \in (0,1)$ is a decay factor, and $K$ is a small hop limit, tuned on source-domain validation data only. This is the structural answer to the zero-default problem of lexicons.
Then we fuse and train. The concatenation of $\tilde r_v$ with the node's learned embedding is fed to two backbones. The first is the FGVulDet edge-aware graph network over code property graphs. The second is GraphCodeBERT with the rule vector projected and added to its data-flow node embeddings. We train only on Juliet labels, with token and identifier channel dropout applied to source graphs.
Finally, before any detector is trained, we run a channel-shift diagnostic. For each input channel (token embeddings, rule vectors, graph-structural statistics), we measure the maximum mean discrepancy between Juliet and REVEAL. Maximum mean discrepancy is a kernel distance between two distributions. We also measure how well a linear probe on that channel alone predicts vulnerability in each domain. The whole method depends on the rule channel being both less shifted and still predictive on real code. If it is not, the proposal stops there with a clean negative result.
Baselines
The naive reference point is a detector trained on Juliet and evaluated on REVEAL with no intervention. We run it with both backbones: an edge-aware graph network without rule features, and fine-tuned GraphCodeBERT. This is the collapse Chakraborty et al. document. The file gives only relative figures for it: drops of "more than 50%," and REVEAL's "up to 33.57%" precision and "128.38%" recall gains over prior models. No reported absolute Juliet-to-REVEAL F1 was found on Lacuna, so we will measure it ourselves. The paired upper bound is the same model trained on REVEAL's own training split. The gap between these two numbers defines the quantity we try to close.
The strongest existing method for transfer without target labels is MNCRI from Dong et al.. Its best reported result is an F1 of 58.21% on FFmpeg to QEMU with CodeT5, a 5.41% gain over the prior best. No reported result was found on Lacuna for MNCRI on Juliet to REVEAL. We will run it with the authors' recipe on the same backbones, the same unlabeled REVEAL training functions, and the same hyperparameter search budget as our method. We also reproduce the FFmpeg-to-QEMU setting so our implementation can be checked against the 58.21% figure. Our method also uses no target labels. It is also orthogonal to MNCRI, so the combination is evaluated too.
The strongest supervised in-domain reference is FGVulDet, which reports F1 65.63 on CWE-120 versus 44.34 for CodeBERT. It sets the scale of what an edge-aware graph model achieves with real labels.
Two ablations isolate the key ingredients. The first moves the same rule vectors from the statement nodes to the graph level, concatenated after pooling. This is the sentence-level variant that lingCNN found weaker, and it tests whether node-level placement matters. The second removes the data-flow spreading step, keeping raw $r_v$ only, which tests whether structure fixes the lexicon's zero-default weakness. A third ablation removes token dropout. Beating the baselines means a statistically reliable gain over the naive reference and over MNCRI in vulnerable-class F1 and precision-recall area on REVEAL across five seeds, at matched compute. The gain should also shrink when analyzer features are randomly permuted across nodes, which rules out gains due only to the extra parameters.
Why not just…?
Why not just feed the analyzer's verdict to a stacking classifier at the end? This is the most serious threat. D2A's 70–80% false-positive reduction shows that late use of analyzer output already carries a strong signal. Our graph-level ablation is exactly this late-fusion model trained on the same features. If it matches the node-level version on REVEAL, the structural part of our claim is false. We would then report that the lexicon helps transfer but graph placement does not.
Why not just rename identifiers and normalize Juliet's boilerplate? If the shortcut is token artifacts, deleting the artifacts might be enough. We include canonical identifier renaming plus de-templating of Juliet as a baseline. We expect it to remove the variable-name shortcut but not the deeper template regularities in control flow, which neither channel dropout nor renaming touches but rule features largely ignore. If normalization alone closes most of the gap, we will say so.
Why not just fine-tune a large pretrained code model on the raw tokens? The file's evidence argues against expecting this to transfer. Token models score high on synthetic suites and then collapse (Chakraborty et al.). A pure LLM reaches only 0.11 precision without symbolic grounding (MoCQ). Fine-tuned GraphCodeBERT is our naive reference, so this alternative is measured directly rather than dismissed.
Why not just train on real labeled data such as D2A instead of synthetic data? Where real labels exist, that is the right choice. D2A's labels, however, still carry up to 47% noise, and many languages and vulnerability classes have no such corpus. Our target is the common case where synthetic suites are the only clean labels. We also report a variant that adds D2A pretraining, to show whether the lexicon still helps once real data is available.
Proposed Experiments
The main experiment trains on Juliet C/C++ functions and tests on the REVEAL dataset (about 9% vulnerable). We report vulnerable-class precision, recall, F1, and precision-recall area, because accuracy is meaningless at that class balance. The decisive checks come in four stages.
The first check is the channel-shift diagnostic described above. It must show that rule vectors drift less than token embeddings while remaining predictive on REVEAL.
The second is the full method versus the naive, normalization, MNCRI, and late-fusion baselines. The test passes only if node-level, propagated rule features beat late fusion and beat MNCRI on the transfer gap.
The third is a shortcut test. We apply identifier renaming to the REVEAL test set, following the perturbation concern raised in Adversarial Attacks on Code Models with Discriminative Graph Patterns, and compare how much each model drops. It also includes a BUGFARM-style subtle-bug evaluation, which checks whether reliance on rules creates new blind spots for bugs that match no rule. We expect this to be the method's honest weakness, and we will measure it rather than hide it.
The fourth is a secondary transfer to D2A. Here Infer is removed from the feature set, because D2A labels come from Infer and keeping it would make the evaluation circular. We also run FFmpeg to QEMU to place our numbers next to the reported 58.21%.
A positive outcome would establish static-analyzer output as a reusable, domain-stable input channel for code models. A negative outcome at the diagnostic stage would also be informative. It would show that analyzer behavior itself differs between synthetic and real code, which would redirect effort toward making synthetic suites more realistic.
References
- Deep Learning Based Vulnerability Detection: Are We There Yet?
- A Linguistically Informed Convolutional Neural Network
- Sentiment analysis based on improved pre-trained word embeddings
- VulSniper: Focus Your Attention to Shoot Fine-Grained Vulnerabilities
- Enhancing Code Vulnerability Detection via Vulnerability-Preserving Data Augmentation
- Challenging Bug Prediction and Repair Models with Synthetic Bugs
- Joint Geometrical and Statistical Domain Adaptation for Cross-domain Code Vulnerability Detection
- Adversarially Masking Synthetic to Mimic Real: Adaptive Noise Injection for Point Cloud Segmentation Adaptation
- D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis
- Automated Static Vulnerability Detection via a Holistic Neuro-symbolic Approach
- GraphCodeBERT: Pre-training Code Representations with Data Flow
- Adversarial Attacks on Code Models with Discriminative Graph Patterns
- Adversarial Attacks on Code Models with Discriminative Graph Patterns