Authors: Michael Roth, Shyam Upadhyay
Venue: NAACL-HLT 2019
Abstract
Standard word embeddings often confuse synonyms with antonyms because both types of words appear in similar contexts. This paper introduces a method to distinguish them in low-resource languages by combining cross-lingual transfers with a simple but powerful linguistic intuition: antonyms frequently co-occur with contrastive discourse markers like "but" or "however."
Introduction
In Natural Language Processing (NLP), distributional semantics—the idea that words with similar meanings appear in similar contexts—is a cornerstone. However, this approach has a "blind spot" regarding antonymy. Because words like "hot" and "cold" are both used to describe temperature or weather, their vector representations (embeddings) end up very close to each other, making it difficult for models to tell them apart. While high-resource languages like English have extensive labeled datasets to help models learn this distinction, most of the world's languages do not. The authors aim to bridge this gap by using monolingual information available in the target language—specifically, how words interact with discourse markers.
Methodology: The LingTrans System
The authors propose LingTrans, a system designed to transfer lexical relation knowledge from a high-resource source (English) to lower-resource target languages.
The system consists of three main stages:
- Translation Module: Instead of a single best translation, the system identifies the "n-best" translations using
vecmap(an unsupervised alignment tool). This prevents the "translation convergence" problem where distinct English words (e.g., "more" and "less") might initially map to the same target word. - Re-ranking via Discourse Markers: To select the most accurate translation pair, the model utilizes contrastive discourse markers (e.g., although, however, but). It calculates Pointwise Mutual Information (PMI) to identify word pairs in the target language that frequently appear near these markers, signaling antonymy.
- Classification: These refined translations train a LexNet model—a feed-forward neural network that uses word embeddings to classify the final lexical relation.
Experimental Setup
The approach was evaluated across four languages with varying resources:
- English: Primary training source (10,932 pairs).
- German: Development and tuning (811 pairs).
- Vietnamese: Evaluated using the ViCon dataset (353 pairs).
- Hindi: Evaluated using a newly crawled dataset from
hindi2dictionary.com(1,000 pairs).
Monolingual data for discourse marker analysis was sourced from Wikipedia (German), LORELEI (Vietnamese), and HindEnCorp (Hindi).
Results and Analysis
The LingTrans model significantly outperformed baselines that relied solely on direct translation (BestTrans) or cross-lingual mapping without discourse cues (NoTrans).
As shown in the results table, the inclusion of linguistic features led to substantial gains, particularly in Vietnamese, which saw a ~14-point absolute improvement in F1-score. This suggests that while cross-lingual embeddings provide a foundation, target-language-specific signals are essential for capturing fine-grained semantic nuances. The authors also noted that semantic properties related to antonymy aren't perfectly preserved across language families, making these local signals even more critical.
Conclusion and Future Work
The study demonstrates that discourse markers serve as a robust semantic filter, especially when distributional embeddings fail. However, limitations remain: approximately 30% of errors were attributed to data sparsity for infrequent words. The model also struggled with complex morphological markings and differences in linguistic register. Future work could extend this framework to other lexical relations, such as using markers like "specifically" to identify hypernymy (is-a relationships) in low-resource settings.
Summary updated