Generative Data Augmentation for Class Imbalance

62 concepts55 papers1 proposal

When training machine learning models, we frequently encounter the "needle in a haystack" problem: the target categories we care about most—such as online fraud, rare medical conditions, or specific linguistic nuances—are severely underrepresented in the training data.

Traditionally, practitioners fixed this class imbalance using geometric oversampling techniques like SMOTE, which interpolate new data points between existing minority samples in a continuous feature space. However, for discrete, highly structured data like natural language, geometric interpolation fails. If you average the vector representations of the words "cat" and "dog," you do not get a meaningful intermediate word; you get semantic noise. As Bansal et al. observe, naive oversampling often introduces destructive artifacts that degrade downstream classifier performance.

To solve this, a powerful paradigm has emerged: Generative Data Augmentation. Instead of manipulating vectors geometrically, researchers use generative models to navigate the true semantic manifold of language, synthesizing high-quality, grammatically sound training examples that fill the gaps in imbalanced datasets.

Why does generative augmentation work where traditional methods fail? The core intuition lies in a generative model's ability to understand context, syntax, and intent. Rather than duplicating text or randomly swapping words, generative models perform semantic augmentation (Ma et al.), producing variations that are diverse yet label-consistent.

Early iterations of this idea focused on localized structural expansions. For instance, Jain et al. utilized recurrent language models to expand on frequent seed phrases found in minority classes, ensuring the synthetic text preserved the target class's semantic structure. In highly structured tasks, Qian et al. applied Generative Adversarial Networks (GANs) to hallucinate valid syntactic shortest dependency paths, allowing models to learn the grammatical skeletons of rare linguistic categories.

With the advent of Large Language Models (LLMs), this process has evolved from structural manipulation to sophisticated text transformation. Modern generative augmentation typically leverages three main mechanisms to balance datasets:

1. Paraphrasing and Back-Translation

The most straightforward way to expand a minority class is to rephrase existing samples. By passing a rare sentence through an LLM or a text-to-text transformer, the system introduces linguistic variety while preserving the original semantic intent.

  • Bhatt et al. demonstrated this by using text-to-text models to paraphrase condescending language, significantly outperforming simple oversampling.
  • Alam et al. combined paraphrasing with back-translation to expand the diversity of rare, check-worthy instances in multimodal datasets.
  • Fox extended this concept to multimodal document classification, pairing natural language paraphrasing with visual noise transformations to rescue models from extreme class imbalances.

2. Controlled Counterfactual Generation

Instead of just multiplying minority examples, models can actively transform majority examples into minority ones by flipping specific attributes. This creates precise decision boundaries for downstream classifiers.

  • Sentiment and Style Flipping: Zhang et al. used LLMs to generate counterfactual negative reviews, balancing heavily skewed, positive-biased opinion datasets without requiring manual labeling.
  • Controlled Rectification: In fact-checking, Rohrbach used LLMs to generate rectified versions of false claims based on expert ground truth, balancing the dataset while maintaining realistic stylistic patterns.
  • Grammatical Alignment: Piergentili et al. leveraged LLMs to generate synthetic triplets consisting of masculine, feminine, and neutral versions of the same sentence, providing the volume of balanced examples needed to train gender-neutral machine translation systems.

Few-Shot Synthesis and Recombination

When the minority class is so scarce that there are not enough examples to paraphrase, LLMs can act as few-shot training example generators (Muresan et al.). By prompting an LLM with just a handful of seed examples, the model leverages its massive pre-trained prior knowledge to synthesize entirely new data points for underrepresented categories (Kim et al.; Galassi et al.).

+------------------------+     +------------------------+     +------------------------+
|  Few Seed Examples     | --> | Large Language Model   | --> | Diverse Synthetic Data |
|  (e.g., Rare Fallacy)  |     | (Leveraging Priors)    |     | (Balanced Dataset)     |
+------------------------+     +------------------------+     +------------------------+

To prevent the LLM from simply repeating the seeds, researchers introduce structural recombination. For example, Kan et al. broke down existing high-quality examples into a library of core components (triggers and arguments), randomly recombined them into novel schemas, and then tasked an LLM with synthesizing coherent natural language sentences around those new combinations. This guarantees that the generated data is structurally novel yet logically sound.

The Curation Filter: Managing Generative Noise

While generative models are highly capable, they are not infallible. They can hallucinate, introduce semantic drift, or generate examples that fail to truly represent the target minority class. If left unchecked, this synthetic noise can pollute the training set and degrade model performance.

To combat this, modern generative augmentation pipelines implement a generate-then-filter architecture.

[ Seed Data ] ---> [ LLM Generator ] ---> [ Curation / Filter ] ---> [ Downstream Classifier ]
                                                 |
                                         (Removes Noise &
                                          Semantic Drift)

The curation step acts as a quality control gatekeeper:

  • Similarity and Core Preservation: Lu et al. filtered generated text using similarity metrics to ensure that the emotional core of an utterance was preserved during the rephrasing process.
  • Learning Dynamics: In ultra-low-data regimes, Seedat et al. introduced a curation filter based on model learning dynamics to weed out noisy synthetic tabular generations before they could misinform the classifier.
  • Teacher-Student Frameworks: Lu et al. utilized a conditional teacher-student training strategy when generating synthetic clinical notes with GPT-2, systematically mitigating the impact of inherent text generation noise on high-stakes patient outcome predictions.

By pairing the creative, distribution-filling capabilities of generative models with rigorous discriminative filters, generative data augmentation effectively bridges the gap between data-hungry deep learning architectures and the naturally skewed, long-tailed distributions of real-world data.

Key Foundational Work

Event Factuality Identification via Generative Adversarial Networks with Auxiliary Classification (2018) — contributed 2 concept elements to this direction.
Querying Across Genres for Medical Claims in News (2020) — contributed 1 concept elements to this direction.
A Multi-Task Approach to Open Domain Suggestion Mining (Student Abstract) (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_generative_data_augmentation_synthetic_d_fcd0579c,
  title = {generative data augmentation (synthetic data generation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-data-augmentation-synthetic-data-generation/rdr_fcd0579c124146ecb98415e125604b13},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_generative_data_augmentation_synthetic_d_fcd0579c_v1,
  title = {generative data augmentation (synthetic data generation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-data-augmentation-synthetic-data-generation/rdr_fcd0579c124146ecb98415e125604b13/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake