Knowing Which Kind of Unfamiliar: Selective Vulnerability Detection with an Induced API Lexicon

Learned vulnerability detectors look strong on benchmark splits and then lose more than half their performance on real projects. The reason is that they latch onto function and variable names rather than program logic (Deep Learning based Vulnerability Detection: Are We There Yet?). One response is selective prediction: the detector answers only when it is likely to be right and sends the rest to a human security auditor. Abstention only helps if the uncertainty score points at the right functions. On new code, almost every function is unfamiliar in some way. It may call functions the model has never seen, or it may combine control and data flow in patterns the model has never seen. These are different kinds of unfamiliarity, and only some of them should trigger abstention.

We propose to separate them. We automatically build a lexicon of API behaviour by clustering how each called function is used in a large body of unlabeled C code: which kinds of arguments it takes, whether it writes through a pointer, and whether its result is checked or used as an index. This is the same move that let computational linguists induce verb classes from syntactic usage patterns instead of hand-built dictionaries. With that lexicon, an unseen callee that behaves like a known memory copier is no longer "novel." The detector can then abstain only on functions whose structure is genuinely new or whose callees cannot be placed in any known class. The claim is that this targeted abstention gives lower error at a fixed coverage than existing hybrid uncertainty methods applied directly to graph-based code models.

Core Claim

Hybrid uncertainty quantification combines two kinds of doubt. Aleatoric doubt is ambiguity in the data itself. Epistemic doubt is the model's ignorance, usually measured as distance from the training data. The method of Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks converts both kinds of score into percentile ranks. It relies on softmax confidence for in-distribution inputs and mixes in Mahalanobis distance for out-of-distribution ones. It reports a 13.0% reduction in area under the risk-coverage curve on 20 Newsgroups. The same paper also reports the failure mode that matters here: under severe covariate shift the hybrid advantage shrinks, because epistemic distance swamps everything else.

Cross-project vulnerability detection is exactly that regime. We hypothesize that the swamping is mostly caused by lexical novelty, meaning new identifiers and APIs, and that lexical novelty is a poor predictor of error once each callee's behaviour is known. Our claim is concrete. We decompose epistemic uncertainty on a code property graph (a single graph that merges a program's syntax tree, control flow and data flow, as in Devign) into two parts:

  • Structural novelty, measured on a graph whose callee identifiers are replaced by their induced lexicon classes.
  • Unresolved-callee novelty, meaning callees whose usage patterns fit no lexicon class.

Dropping raw identifier novelty from the abstention score should lower selective risk under cross-project shift, compared with applying hybrid uncertainty to standard graph-network embeddings.

Research Program

Building the lexicon. We first build what we call an API frame lexicon. For every called function in a large unlabeled C corpus, we extract a usage frame from the code property graph. A usage frame is the distribution over the following features at its call sites:

  • argument kinds: pointer, integer that flows from a length computation, constant string;
  • data-flow roles: argument written through, return value compared against null, return value flowing into an array index or allocation size;
  • control context: inside a loop, or dominated by a bounds check.

This mirrors subcategorization frames, the record of which grammatical complements a verb takes. Inducing German Semantic Verb Classes from Purely Syntactic Subcategorisation Information clustered such frames into semantic verb classes, reaching mutual information 0.372 against 0.229 for random seeding.

That work constrains our design in two ways:

  • Hard k-means assigns each verb to a single class, which fails for polysemous verbs. We therefore use a mixture model, so a function like realloc can belong to several classes at once.
  • Finer frames improved separation but made rare predicates sparse. We therefore back off from fine to coarse frame features when a callee has few call sites.

The lexicon needs no documentation. That matters because the documentation-driven route of API Pack does not cover project-internal helpers, and the symbolic behaviour synthesis in M3: Semantic API Migrations covered only 37% of functions.

Building the detector. The detector is a gated graph network over code property graphs, following the Devign/REVEAL design. Each callee node carries both its identifier embedding and its lexicon class-membership vector. Each test function receives three scores:

  1. Aleatoric: softmax response.
  2. Structural epistemic: Mahalanobis distance of the graph embedding, computed by a second readout that sees only class memberships, not identifiers.
  3. Unresolved-callee: the largest negative log-likelihood, under the lexicon mixture, of any callee's frame.

We combine the three with the percentile-rank scheme of the hybrid method. One change addresses that method's threshold fragility: the mixing weights and the boundaries between in-distribution and out-of-distribution regions are fit on a held-out project, not a held-out random split. A useful side effect is that the frame likelihood also absorbs some static-analysis extraction noise, the code analogue of the parser "pollution" Schulte im Walde observed. This is a secondary question we will measure, not a claim.

Baselines

Naive reference: softmax confidence on the graph network. We train the same detector, rank functions by softmax confidence, and abstain on the lowest-ranked. Every method must beat this reference to justify any added machinery.

Strongest existing competitor: hybrid uncertainty on graph embeddings. We apply the method of Hybrid Uncertainty Quantification for Selective Text Classification in Ambiguous Tasks unchanged to the identifier-aware graph network's embeddings. This is the obvious combination of the input directions. It is also the method most likely to already work, since on text it matched deep ensembles at under 5% overhead. Its selection rule is tuned on the same held-out project with the same search budget as ours. Beating it means a statistically reliable reduction in area under the risk-coverage curve on the cross-project test set, across five seeds.

Deep ensembles. We train five graph networks with predictive-entropy abstention. They get the same total training compute per member as our model.

Sequence-model reference. CodeBERT is fine-tuned on the CodeXGLUE defect detection task and uses softmax abstention. Its reported forced-prediction accuracy there is 62.08% (CodeXGLUE). For forced prediction on the graph side, Devign reports 74.11% accuracy on 40 recent CVEs. REVEAL reports up to a 33.57% precision boost and a 128.38% recall boost over prior models on its Chromium/Debian data.

No reported result was found on Lacuna for risk-coverage or any selective metric on Devign, CodeXGLUE defect detection, or REVEAL data. We will therefore produce all selective numbers for every baseline ourselves, under identical splits.

Ablations. Four ablations isolate the key ingredient:

  • (a) Identifier-only callees, with no lexicon. This is the hybrid baseline itself.
  • (b) The lexicon added as model features, but abstention still using identifier-aware distance. This separates "the lexicon makes the model better" from "the lexicon makes abstention better."
  • (c) Hard k-means classes instead of the mixture.
  • (d) The unresolved-callee score removed.

Why not just…?

Why not just anonymize identifiers, replacing every name with a placeholder? This removes lexical novelty with no lexicon at all, and it is a real threat. It also deletes the information that strcpy and printf differ, which matters for memory-safety bugs. We include an anonymized graph network with hybrid uncertainty as a baseline. If it matches our method's risk-coverage curve, the lexicon is unnecessary, and we would reframe the work as a negative result about identifier-aware uncertainty.

Why not just use deep ensembles? In the hybrid uncertainty paper, ensembles cost several full models and were matched by a single-model hybrid score. Ensemble disagreement also collapses when every member makes the same default guess in an unobserved region (Density-aware NeRF Ensembles). Cross-project code plausibly produces that effect, because all members can confidently key on the same spurious names. The ensemble baseline tests this directly.

Why not just use API documentation, as API Pack does, instead of inducing a lexicon? Documentation exists for public libraries, not for the project-internal helpers that dominate Chromium-scale code. Even with documentation and retrieval, unseen-API accuracy reached only 49.5% for fine-tuned CodeLlama-13B in API Pack. We will report what fraction of test-set callees have any documentation. We will also compare against a lexicon variant seeded only from documented libc functions.

Proposed Experiments

Training and test data. We train on the Devign corpus. The primary test is the REVEAL Chromium/Debian data, which is de-duplicated and roughly 9% vulnerable. The secondary test is leave-one-project-out within Devign's four projects (Linux, QEMU, Wireshark, FFmpeg).

Metrics. Because of class imbalance, we report three measures:

  • area under the risk-coverage curve, with risk defined as balanced error;
  • selective precision and recall at 30%, 50% and 70% coverage;
  • the fraction of truly vulnerable functions routed to auditors.

Lexicon quality. We evaluate the lexicon against a small, hand-labeled gold set of about 150 standard C library and common project functions, grouped by behavioural role. We report purity and mutual information, in line with Schulte im Walde. This gold set is our own construction, because the file contains no existing one.

Decisive check: stratifying test functions. We split cross-project test functions into three groups:

  • (i) all callees seen in training;
  • (ii) unseen callees that the lexicon confidently resolves;
  • (iii) callees the lexicon cannot resolve.

The hypothesis predicts a specific pattern:

  • The identifier-aware hybrid baseline abstains heavily on stratum (ii), even though the detector's error there is close to its error on stratum (i).
  • Our method answers stratum (ii) and concentrates abstention on stratum (iii) and on high structural novelty.

The claim is falsified under either of two outcomes:

  • Error on stratum (ii) is as high as on stratum (iii).
  • The anonymized baseline matches our curve.