Introduction
Translating organization names (ONs) is a significant challenge in Natural Language Processing due to their nested structures, which often include person names, locations, and brand names. Standard statistical machine translation (SMT) models frequently fail because of two primary issues: the Out-of-Vocabulary (OOV) problem, where unique brand names are mis-segmented into meaningless pieces, and the difficulty of retrieving bilingual search results from monolingual queries.
This paper proposes a novel system that leverages the live web to find translation equivalents. By combining a specialized segmentation strategy with a heuristic query generator and an "asymmetric" alignment algorithm, the system significantly improves translation accuracy for complex organization names.
Methodology
System Architecture
The proposed system follows a multi-stage pipeline designed to handle the nuances of Chinese ONs and the variability of web-based data.
1. Chunking-based Segmentation
To prevent the incorrect splitting of OOV brand names, the system first "chunks" the Chinese string into four categories: Location (LC), Name (NC), Modification (MC), and Key word (KC). By identifying the "Name Chunk" first using a Conditional Random Fields (CRFs) model, the system protects unique entities from being fragmented during the subsequent word segmentation phase.
2. Heuristic Query Construction
To bait search engines into returning bilingual pages, the system identifies the most "informative" words in the Chinese name using Mutual Information (MI). These specific words are translated into English and added to the search query. These English "clues" ensure that the search results are highly relevant and likely to contain the target translation.
3. Asymmetric Alignment
The most novel component is the asymmetric alignment process. Instead of attempting to match a Chinese entity to a pre-identified English entity—which relies on error-prone English Named Entity Recognition (NER)—the system slides the Chinese ON across entire English sentences found in web snippets.
Using an extended Kuhn-Munkres (KM) algorithm, the system calculates an alignment score for every possible fragment to find the optimal word-to-word matching, effectively bypassing the NER bottleneck.
Experimental Setup and Results
Datasets
The researchers utilized the LDC2005T34 corpus, specifically the Indus_corpus (proper names) and Org_corpus (industry names). The training set included 65,835 manually labeled translation pairs, while the testing set consisted of 503 randomly selected pairs.
Key Results
- Overall Performance: The system achieved a 48.71% Top-1 accuracy, a dramatic improvement over the Moses SMT baseline, which only reached 18.29%.
- Alignment Accuracy: The asymmetric alignment method outperformed traditional alignment methods by over 12%, reaching 36.18% precision.
- Segmentation Impact: The chunking-based strategy improved exact match precision from 18.29% to 21.47%, demonstrating the importance of protecting OOV names early in the pipeline.
Conclusion
By integrating chunking-based segmentation with heuristic web mining and asymmetric alignment, this system provides a robust solution for the translation of complex organization names. While it remains dependent on the presence of translations within search engine snippets, it successfully mitigates the cascading errors common in traditional NER-based translation systems.
Summary updated