TL;DR
This paper introduces a unified framework for extracting bilingual terminology, handling both single-word and multi-word terms from comparable corpora. By enhancing existing context-based and neural network approaches, the authors achieve improved accuracy and demonstrate the framework's adaptability to specialized domains.
What Problem Does This Paper Address?
Extracting bilingual terminology, especially for multi-word terms (MWTs), from comparable corpora (texts on similar topics but not direct translations) is a challenging task. Existing methods often treat single-word terms (SWTs) and MWTs separately, requiring specialized linguistic knowledge or relying on predefined syntactic patterns for MWTs. This fragmentation makes it difficult to build a comprehensive bilingual lexicon. Furthermore, specialized domains often have limited comparable corpora, hindering the performance of data-driven systems. This paper aims to bridge this gap by proposing a unified framework that handles terms of variable lengths and incorporates strategies to leverage external data for improved performance in specialized domains.
Methods
The authors propose a unified framework that builds upon and enhances two main approaches:
- Context-Based Projection Approach: This traditional method relies on word co-occurrence statistics within context windows. The paper introduces two key enhancements:
- Distance-Sensitive Co-occurrence (DSC): Penalizes context words that are further away from the target word, assuming they are less relevant.
- Weighted Mutual Information (WMI): Smooths low co-occurrence counts to prevent overestimation and improves the reliability of the statistics.
- Neural Network-Based Approach (Word Embeddings): This approach uses dense vector representations of words. The paper focuses on bilingual word embedding mapping and proposes:
- Renormalization after Mean Centering: Addresses the issue that mean centering can de-normalize word vectors, improving bilingual mapping performance.
- Concatenation for Usage of External Data: To overcome data scarcity in specialized corpora, they propose concatenating word vectors trained on specialized corpora with those trained on larger general corpora. This preserves specialized features while benefiting from broader linguistic knowledge.
These methods are then integrated into a unified framework for both SWT and MWT extraction. For MWTs, a novel representation is proposed where the vector for an MWT is the sum of the distributional representations of its composing words, allowing for variable-length term alignment.
Datasets
The experiments were conducted using the following datasets:
- Comparable Corpora:
- Breast Cancer Corpus (BC): Specialized medical domain corpus.
- Wind Energy Corpus (WE): Specialized technical domain corpus.
- News Commentary (NC): General domain corpus used as external data.
- Bilingual Lexicon:
- French/English Dictionary (ELRA-M003376): Used for French/English experiments.
- Italian/English Seed Lexicon: Used for Italian/English experiments.
- Pre-trained Word Embedding Models:
- Models trained on large general corpora (e.g., UKWAC, Wikipedia, BNC) for English.
- Models trained on Italian corpora.
- Models trained on French corpora using the skip-gram model.
Key Insights and Ideas
- Unified Framework is Crucial: Treating SWTs and MWTs separately is suboptimal. A unified approach simplifies the process and potentially improves overall performance.
- Context Matters, Distance Matters More: The intuition that words further away from the target word are less relevant is empirically validated, leading to improved SWT extraction with the Distance-Sensitive Co-occurrence (DSC) method.
- Data Augmentation is Key for Specialized Domains: For low-resource specialized domains, leveraging external general-domain data (via concatenated word embeddings) significantly boosts performance, without overly biasing the specialized domain's features.
- MWT Representation Simplification: Representing MWTs as the sum of their word vectors simplifies alignment and allows for handling variable-length terms efficiently.
- Enhancements are Mutually Beneficial: Combining WMI and DSC for SWT extraction yields better results than either alone, suggesting their complementary nature. Similarly, combining various neural network enhancements (L2, MC, Orth., Renorm., Concat.) leads to the best MWT extraction results.
Key Results
- SWT Extraction: The proposed enhancements (WMI and DSC) improved the standard approach (SA) and selective standard approach (SSA) results. For example, in the BC corpus, SA + WMI + DSC achieved 29.5% accuracy, an improvement over the baseline of 25.9%. The neural network-based approaches, especially when combined with external data and renormalization, achieved higher accuracies, with the proposed method reaching 83.2% and 83.1% for BC and WE respectively.
- MWT Extraction: The proposed CMWEP method, particularly when incorporating external data (concat) and renormalization, significantly outperformed other approaches. The best result achieved was 73.4% MAP and 61.6% accuracy on the WE corpus, a notable improvement over the baseline CMCBP + SSA (66.0% MAP).
Limitations and Future Work
- Limitations:
- The paper acknowledges that the compositional approach for MWTs can struggle with terms where the overall meaning is not a simple combination of its parts (weak compositionality).
- Ambiguous translations and word order are identified as sources of errors, particularly for MWTs. The current method ignores word order to maintain manageability for variable-length terms.
- The performance of the proposed framework for languages beyond French/English and Italian/English is yet to be explored.
- Future Work:
- Restructure MWT representation to account for varying contributions of notional words.
- Investigate the framework's performance with other language pairs.
- Develop methods for word disambiguation when multiple translations are possible.
Summary updated