Introduction
Bilingual Lexicon Induction (BLI) is the task of automatically constructing dictionaries between two languages using only monolingual text data. Modern approaches typically map word embeddings into a shared vector space, where translation is treated as a nearest-neighbor search. However, these methods often suffer from source-side bias, a phenomenon where proximity in high-dimensional space is not mutual.
As shown in Figure 1, the closest target-language word to the English word "apple" might be the French word for "cherry." Yet, from the target perspective, "cherry" might correctly point back to "cherry" in the source language. This asymmetry leads to "hubs" that attract many incorrect source words, significantly degrading the model's precision.
Methodology: Bi-directional Retrieval
To address this bias, the researchers propose a retrieval framework that treats translation as a "two-way street." Instead of relying solely on a one-way source-to-target similarity search, the model incorporates a reverse verification step.
The proposed method follows a three-step process as illustrated in Figure 2:
- Candidate Generation: Identify the top $n$ target-language candidates for a given source word using standard similarity measures.
- Reverse Verification: For each candidate, perform a "back-retrieval" to determine the rank and similarity of the original source word from the target word's perspective.
- Fusion Scoring: Combine the forward and backward scores using a weighted hyperparameter $\lambda$ to produce a final ranking.
Two specific algorithms were introduced: BRTR (Back Retrieving of Top Rank), which re-ranks based on reciprocal ranks, and BRSR (Back Retrieving of Similar Rank), which utilizes raw similarity scores for finer adjustment. This approach is "plug-and-play" and can be added to existing BLI models without retraining the underlying embeddings.
Experimental Results
The framework was evaluated on the MUSE dataset across six language pairs, categorized into similar (e.g., EN-FR, EN-ES) and distant (e.g., EN-ZH, EN-JA) pairs. The results demonstrate that bi-directional retrieval consistently improves performance across various base models like MUSE and VECMAP.
Key findings include:
- Significant Gains in Distant Languages: The method is particularly effective for difficult language pairs. For English-Chinese (EN-ZH), the approach improved the MUSE baseline by 8.6% in Precision@1.
- Universal Applicability: The geometric "back-check" proved effective in both supervised and unsupervised settings, confirming it addresses a fundamental property of aligned embedding spaces.
- Hyperparameter Sensitivity: The authors found that for distant language pairs, the backward check (controlled by $\lambda$) is even more critical for accuracy than it is for similar pairs.
Conclusion
By formalizing and mitigating source-side bias through bi-directional verification, this research provides a robust, model-agnostic enhancement for bilingual lexicon induction. While the study focused on static embeddings, the authors suggest that future work could involve applying these geometric insights to contextual embeddings and Large Language Models (LLMs).
Summary updated