Introduction: The Challenge of Low-Resource Text Classification
Deep learning models, particularly Transformer-based architectures like BERT, require vast amounts of labeled data to achieve high performance. In "low-resource" scenarios—where only a handful of labeled examples per class are available—performance often degrades significantly. Data Augmentation (DA) is a common strategy to mitigate this, but traditional techniques frequently suffer from a "hallucination" problem where the generated text loses its original meaning or logical coherence.
Simple methods like Easy Data Augmentation (EDA) may randomly swap or delete words, often resulting in nonsensical sentences. For instance, a request to "Add a song by Bruno Mars" might be transformed into "Add a song Bruno Mars by Uptown Funk," which is semantically incoherent. Even advanced methods like ALP (Augmented Language Patterns) can struggle to maintain the specific context of keywords. GDA aims to solve this by ensuring augmentation is context-aware and regulated.
The GDA Methodology
Grammar-based Data Augmentation (GDA) addresses these issues by using "slot information"—the semantic anchors of a sentence—to build formal grammar rules for generating new training data. By manipulating these rules rather than just swapping random words, GDA produces augmented text that is both syntactically diverse and semantically accurate.
The GDA pipeline consists of three primary phases:
- Rule Inference: The system identifies keywords (slots) in a sentence and replaces them with variables. For example, the sentence "Play Yesterday by The Beatles" is abstracted into a Rule of Grammar (RoG):
Play $song by $artist. - Rule Manipulation: This is the core innovation of GDA. The system expands these rules using two strategies:
- Distance-based: A normalized edit-distance algorithm identifies similar sentence structures and merges them into flexible rules (e.g., merging "Add $song to playlist" and "Insert $song into playlist").
- Keyword-based: Rules sharing the same slots are grouped to find recurrent patterns in how users express the same intent.
- Sentence Generation: The system samples from these generalized grammars and populates the slots with valid entities to create thousands of new, semantically sound training examples.
Experimental Evaluation
The authors evaluated GDA across a variety of datasets, categorized by whether they contained native slot information (like ATIS and Snips for voice assistants) or required Named Entity Recognition (NER) to discover slots (like IMDB and AGNews).
Key Results
- Performance Gains: In extreme low-resource settings (e.g., 5-shot experiments), GDA consistently outperformed state-of-the-art methods. On the SST-2 sentiment dataset, GDA improved prediction accuracy by 19.38% over the previous best method (ALP).
- GDA vs. Large Language Models (LLMs): Surprisingly, GDA often outperformed LLMs like Bard and fine-tuned BERT for data generation. This suggests that GDA's grounding in the specific distribution of the small training set is more effective than the broad, sometimes irrelevant, knowledge introduced by LLMs.
- Explainability: Unlike "black-box" LLM-based augmentation, GDA’s rules are human-readable, allowing researchers to understand exactly why a specific sentence was generated.
Conclusion and Future Work
GDA demonstrates that anchoring data augmentation around semantic slots prevents the semantic drift common in other methods. While highly effective, GDA's performance is currently dependent on the presence of clear "slots" or keywords. Future work may focus on identifying "implicit slots" in more general text or combining GDA’s structural rigors with the linguistic diversity of LLMs to fill templates with even more varied vocabulary.
Summary updated