Overcoming Data Scarcity in Software Vulnerability Detection

40 concepts38 papers1 proposal

Deep learning has transformed software engineering, yet applying it to software security introduces a fundamental bottleneck: vulnerability data scarcity. While open-source repositories contain billions of lines of functional code, verified examples of security flaws are remarkably rare.

To train a model to distinguish between secure and vulnerable code, standard supervised learning requires vast, diverse, and precisely labeled datasets. In the real world, however, security data is a "data desert." This scarcity stems from several structural challenges:

  • High Cost of Expertise: Labeling a vulnerability requires deep security domain knowledge, making manual curation unscalable.
  • Noisy Ground Truth: Public code repositories and security commits are frequently mislabeled or missing critical context, as noted by Wang et al..
  • Compilation Failures: For binary-level analysis, models require executable code. However, as Kim Le et al. point out, many open-source vulnerability disclosures provide only isolated code fragments that cannot be successfully compiled.

When models are trained on these small, narrow datasets, they suffer from a catastrophic drop in performance when deployed on entirely new, unseen software projects, a phenomenon documented by Chen et al.. To bypass this data drought, researchers have developed two core strategies: semantics-aware data synthesis and data-efficient learning architectures.


The Realism Gap in Synthetic Data

A naive solution to data scarcity is to generate artificial bugs. However, traditional automated bug injection creates a "realism gap."

Bundt et al. evaluated synthetic bugs over 80 CPU-years of testing and discovered that they are significantly easier to find than organic vulnerabilities. Artificial bugs tend to cluster near a program's "main path" of execution, whereas real-world vulnerabilities hide deep within complex, conditional program logic. If a model trains purely on naive synthetic bugs, it learns superficial patterns rather than the intricate logical flaws that characterize actual security risks.

To bridge this gap, modern synthesis frameworks leverage program analysis to inject vulnerabilities into realistic locations. For instance, Nong et al. introduced VGX, a framework that uses a semantics-aware Transformer to understand how data flows through a program. By combining this data-flow tracking with human-refined bug patterns, it injects vulnerabilities that mimic real-world software flaws closely enough to help detectors uncover new zero-day vulnerabilities in production systems like the Linux kernel.

Similarly, in specialized domains like web applications or malware analysis, researchers use structured generation to populate the data desert. Cao et al. combined program slicing with Large Language Model (LLM) synthesis to generate realistic PHP vulnerability patterns, while Singh et al. utilized Generative Adversarial Networks (GANs) to synthesize high-quality malware images for classification tasks where real samples are restricted due to privacy and security sensitivity.


Maximizing the Value of Limited Labels

When high-quality labels cannot be scaled up, the alternative is to alter the learning paradigm so the model requires less data to generalize.

Richer Supervisory Signals via Contrastive Learning

Standard vulnerability detection models are trained using binary classification: a code snippet is labeled either 0 (clean) or 1 (vulnerable). Li et al. argue that binary labels provide an insufficient supervisory signal. Because benign functional code and vulnerable code are often structurally identical except for a single token, a binary loss function fails to teach the model why the code is flawed.

To solve this, the authors developed CLeVeR, a framework that applies multi-modal contrastive learning to software security. Instead of learning from binary labels alone, the model aligns source code representations with natural language descriptions of the vulnerability (such as Common Weakness Enumerations, or CWEs). By using the text as a semantic filter, the model learns the underlying security concepts, allowing it to achieve high detection accuracy and zero-shot bug classification without requiring massive labeled code datasets.

Standard Supervised Learning:
[ Code Snippet ] ---------> ( Binary Loss: 0 or 1 ) ---> Weak Conceptual Understanding

Contrastive Learning (CLeVeR):
[ Code Snippet ] ---------\
                           > ( Shared Latent Space ) ---> Strong Conceptual Understanding
[ Text Description ] -----/

Few-Shot and Zero-Shot Learning

Another approach adapts models to recognize vulnerabilities from only a handful of examples. He et al. introduced Vul-Mirror, which reframes vulnerability detection as a few-shot recognition task—similar to facial recognition. By training the model to measure the distance between code representations rather than classifying them outright, it can identify rare or newly discovered vulnerabilities given only a few documented examples.

In the domain of automated program repair, Xia et al. bypassed the need for historical bug-fix pairs entirely. Traditional repair models require large datasets of "before-and-after" code snippets to learn how to fix vulnerabilities. The authors introduced AlphaRepair, which reframes code repair as a zero-shot "cloze-style" (fill-in-the-blank) task. By leveraging large pre-trained models to predict missing code tokens in a zero-shot setting, they eliminated the dependency on scarce, curated vulnerability-fix datasets.

Data-Efficient Representations

Finally, changing how code is represented can drastically lower data requirements. Massive Transformer models require millions of examples to learn syntax and control flow from raw text. Gao et al. developed DeepDFA, a framework that encodes code semantics directly into a compact "abstract dataflow embedding" before passing it to a Graph Neural Network (GNN). Because the structural rules of program execution are baked directly into the graph representation, the model does not need to waste parameter capacity learning language syntax, making it 75x faster to train and far more data-efficient than text-based models.


Targeted Alignment and Preference Optimization

As the field shifts toward foundational Large Language Models, the challenge has evolved from training detectors from scratch to steering generative models away from producing insecure code.

Even when pre-trained on massive codebases, LLMs frequently generate vulnerabilities because public code repositories are riddled with insecure patterns. To correct this behavior without massive security datasets, researchers use targeted alignment strategies.

Hasan et al. address the scarcity of paired secure/insecure code by introducing a pipeline called DiSCo to distill preference pairs from LLMs. They couple this with Localized Preference Optimization (LPO), an algorithm that focuses the model's loss function strictly on the specific tokens causing a vulnerability. By isolating the exact point of failure rather than calculating loss across an entire file, the model maximizes the learning utility of each scarce data point, allowing smaller models to outperform much larger frontier models in secure code generation.

Key Foundational Work

Evidence-Based Security in Aerospace: From Safety to Security and Back Again (2012) — contributed 2 concept elements to this direction.
DeepBugs: a learning approach to name-based bug detection (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_vulnerability_data_scarcity_e0e26cf2,
  title = {vulnerability data scarcity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/vulnerability-data-scarcity/rdr_e0e26cf24daf4f6683d0fc9331fa37ec},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_vulnerability_data_scarcity_e0e26cf2_v1,
  title = {vulnerability data scarcity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/vulnerability-data-scarcity/rdr_e0e26cf24daf4f6683d0fc9331fa37ec/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake