Commonsense Knowledge Augmentation for Low-Resource Languages via Adversarial Learning

Bosung Kim, Juae Kim, Youngjoong Ko, Jungyun SeoAAAI 2021 · DOI

Introduction

Artificial Intelligence requires "commonsense"—the fundamental facts about the world that humans intuitively understand, such as the fact that a bank is a place to keep money. In Knowledge Graphs like ConceptNet, this is represented as triples: <bank, CapableOf, keep money>. While English-language resources like ConceptNet contain millions of such triples, most other languages—often referred to as "low-resource" languages in this context—have significantly fewer.

Simply translating English triples is insufficient due to linguistic ambiguity. For instance, "bank" could translate to a financial institution (eun-haeng) or a river bank (duk) in Korean. A translated triple like <river bank, CapableOf, keep money> is logically invalid. Traditionally, validating these translations requires expensive manual labeling. This research introduces an adversarial learning framework to automate this expansion without needing target-language labels.

Methodology: Adversarial Knowledge Augmentation

The proposed framework transfers "validity" patterns from resource-rich English to other languages using a three-stage pipeline:

  1. Candidate Generation: English triples are translated into multiple potential candidates in the target language using standard APIs.
  2. Adversarial Validity Scoring: This core component uses a Feature Extractor ($F$) to map triples into a vector space. A Triple Classifier ($C$), trained on English, identifies valid triples. Crucially, a Language Discriminator ($D$) acts as an adversary, attempting to distinguish between English and target-language features.
  3. Augmentation: By forcing the Feature Extractor to "fool" the discriminator, the model learns language-agnostic representations. This allows the English-trained classifier to accurately score the validity of translated triples in new languages.
Overview of the knowledge augmentation framework, illustrating the pipeline from English triple translation to adversarial validity scoring and final knowledge graph augmentation.

Experimental Setup and Insights

The researchers utilized the 3.4 million triples in English ConceptNet as their primary source. To evaluate the model, they developed a manually labeled Korean benchmark and introduced new test sets for 16 additional languages, including French, Chinese, Arabic, and Swedish.

A key insight of this work is treating cross-lingual transfer as a domain adaptation problem. By explicitly forcing the model to ignore language-specific "noise" through adversarial training, the system captures the underlying logic of a triple. This approach proved superior to standard multilingual models like M-BERT, which can be distracted by linguistic variations.

Key Results

The adversarial model demonstrated exceptional performance, particularly on the Korean benchmark, where it achieved 93.7% precision@1. This high precision allowed for the generation of over 626,000 new, valid Korean triples.

Performance of the augmentation model on the Korean benchmark, showing the trade-off between the number of augmented triples and precision at different ranking thresholds (N).

Beyond Korean, the model outperformed baselines across all 16 test languages. Notably, it showed strong zero-shot capabilities, performing well even in languages where it had no prior labeled examples. In Dutch, the adversarial approach improved accuracy by over 20 percentage points compared to standard M-BERT.

Conclusion and Future Work

While the framework significantly advances commonsense knowledge for low-resource languages, challenges remain for linguistically distant language families like Arabic or Slovenian. Future research will focus on bridging these gaps and refining the feature extractor to handle more diverse linguistic nuances. The researchers have released their augmented Korean data and multilingual test sets to support further community efforts in this domain.

Summary updated

BibTeX
@inproceedings{kim2021commonsense,
  author = {Bosung Kim and Juae Kim and Youngjoong Ko and Jungyun Seo},
  title = {{Commonsense Knowledge Augmentation for Low-Resource Languages via Adversarial Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i7.16793},
}
Figure