TL;DR
This paper presents a novel method for automatically building a Spanish medical lexicon by leveraging an existing Portuguese lexicon. By using "cognate mapping" and comparing context vectors of subword units, they achieve a high accuracy in identifying Spanish equivalents, significantly reducing the manual effort typically required for lexicon creation.
What Problem Does This Paper Address?
Building comprehensive lexicons for specialized domains like medicine is crucial for applications like information retrieval and text mining. However, manually creating and maintaining these lexicons is a time-consuming and error-prone process. This is especially challenging for multilingual scenarios where parallel corpora (texts aligned sentence-by-sentence in different languages) are scarce, particularly for specialized domains. The authors aim to address this by developing a semi-supervised method to acquire a Spanish lexicon from a Portuguese seed lexicon, exploiting the close linguistic relationship between the two languages and using non-parallel corpora.
Methods
The core of their approach is "cognate mapping," a heuristic strategy to identify potential Spanish equivalents of Portuguese subword units (semantically minimal morpheme-style units).
- Subword Lexicon and Morpho-Semantic Indexing (MSI): They first define subwords as the fundamental units for their lexicon, moving beyond traditional word or stemmed word representations. MSI then converts documents into an interlingual representation using language-independent semantic descriptors (MIDs).
- Cognate Candidate Generation: They generate Spanish subword candidates from the Portuguese lexicon by applying string substitution rules derived from a commercial dictionary. This process creates potential cognates, even for similar but not identical spellings (e.g., Portuguese 'mulher' vs. Spanish 'mujer').
- Automatic Semantic Validation: To filter out "false friends" (words that look similar but have different meanings), they employ a context-based validation method. They create context vectors for each subword unit (MID) based on the co-occurrence of other MIDs in a local window within their respective language corpora (Portuguese and Spanish).
- Context Vector Comparison: They then compare the context vectors of Portuguese and Spanish MID candidates using similarity metrics like cosine and city-block. A high similarity suggests that the cognate pair is semantically related and likely a true translation.
Datasets
- Medical Corpora: Spanish and Portuguese medical corpora were compiled from heterogeneous WWW sources.
- Spanish: 2,267,841 tokens, 118,021 types.
- Portuguese: 3,406,589 tokens, 133,146 types.
- Portuguese Subword Lexicon: An existing Portuguese subword lexicon was used as the seed.
- Spanish Affixes List: A manually created list of 842 Spanish affixes.
- Commercial Dictionary: Used to derive string substitution rules for cognate generation.
Key Insights and Ideas
- Subwords as Building Blocks: The paper argues that subwords, rather than full words or stems, offer a more appropriate granularity for representing medical terminology, especially given the complex morphology in this domain.
- Leveraging Linguistic Similarity: The close relationship between Spanish and Portuguese allows for effective cognate identification using simple string-based rules, significantly bootstrapping the lexicon acquisition process.
- Context is King for Validation: Comparing the local context in which subword units appear in non-parallel corpora is a powerful heuristic for distinguishing true cognates from false friends, even without direct translation pairs.
- Trade-off in Validation: Increasing the similarity threshold for context vectors reduces false positives but increases false negatives, highlighting the need for careful parameter tuning by lexicon engineers.
Key Results
- High Cognate Identification Rate: The initial cognate generation process identified 11,206 tentative Spanish cognates, representing 79% of the Portuguese seed lexicon.
- Manual Validation Accuracy: Manual validation of a sample revealed an accuracy of 89.4% for the automatically generated cognate pairs, with only 10.6% identified as potential false friends.
- Automatic Validation Effectiveness: The automatic semantic validation using context vectors showed promising results. For instance, using the cosine metric, achieving zero false friends required rejecting 92.2% of the generated MID cognates for manual revision. The city-block metric achieved a similar outcome with a 97% rejection rate.
- Corpus Size Matters: The authors note that for infrequent subword units (MIDs), larger corpora are needed to gather sufficient context evidence for reliable validation.
Limitations and Future Work
- Corpus Size for Infrequent MIDs: The current approach struggles with validating infrequent subword units due to insufficient context evidence in the available corpora.
- Manual Effort Still Needed: While significantly reduced, some manual validation and revision are still necessary, especially for achieving very low false positive rates.
- Future Work: The authors plan to expand the Spanish lexicon by acquiring subwords that cannot be generated from Portuguese translations. This will involve automated stemming and exploring local contexts to find Portuguese translations, requiring significantly larger corpora and potentially clustering techniques to relate documents. The ultimate goal is to evaluate the resulting lexicon in real cross-language information retrieval experiments.
Summary updated