TL;DR
This paper tackles the challenge of translating short, unknown queries in cross-language information retrieval (CLIR) by leveraging the vastness of the World Wide Web. They propose a novel approach that mines bilingual search result pages to automatically extract query translations, offering a flexible and cost-effective alternative to traditional dictionary-based methods.
What Problem Does This Paper Address?
Cross-language information retrieval (CLIR) systems aim to help users find documents in a language different from their query language. A major hurdle in CLIR is translating "unknown queries" – those containing terms not found in standard translation dictionaries. This problem is particularly acute for short queries, which are common in real-world web searches (averaging 2-3 words). Traditional CLIR methods often rely on domain-specific bilingual corpora or dictionaries, which can be scarce, outdated, or lack coverage for specialized terms. The paper highlights that a significant percentage of frequent query terms in real-world logs are not present in existing lexicons, necessitating a more dynamic and scalable translation solution.
Methods
The core of the paper's contribution lies in its search-result-based approach for query translation. This method leverages the Web as a dynamic corpus, specifically by mining bilingual search result pages.
Here's a breakdown of their key techniques:
- Term Extraction: To identify potential translation candidates from noisy search results, they introduce a new association measure called SCPCD. This measure combines symmetric conditional probability (SCP) with context dependency (CD) to better segment terms and identify their lexical boundaries.
- Translation Selection: Once terms are extracted, the paper proposes two methods to estimate the similarity between a source query term and its potential translations:
- Chi-square Method (χ²): A statistical measure based on co-occurrence frequencies of terms in search results. It's computationally simple and relies on page counts obtained from search engines.
- Context-Vector Method (CV): This method represents query terms and their translations as "context vectors" derived from the surrounding words in search result pages. Similarity is then calculated using cosine similarity between these vectors. This approach is more robust for infrequent terms.
- Combined Approach (χ² + CV): Recognizing that the chi-square and context-vector methods have complementary strengths, the authors propose combining them. They also explore integrating this with an anchor-text-based approach (from their prior work) for even better performance. This combined approach uses a linear combination of similarity scores.
Datasets
The paper utilizes several datasets for their experiments:
- NTCIR-2 Title Queries: A set of 50 English title queries from the NTCIR-2 task, used to evaluate translation of short, specific queries.
- Real-world Web Queries:
- Popular Query Set (PE-430/PC-430): 430 frequent English query terms from a real-world Chinese search engine log.
- Random Query Set (RE-100/RC-100): 100 randomly selected English query terms from the same log.
- HK Parallel Corpus: A large English-Chinese text corpus (238,236 paragraphs) used for comparison with traditional corpus-based methods.
- Anchor-text-set Corpus: A collection of 109,416 traditional Chinese web pages with anchor texts containing both Chinese and English terms, used for their anchor-text-based approach.
- Dreamer and GAIS Logs: Two real-world Chinese search engine logs used to collect and prepare test query sets for web query translation experiments.
Key Insights and Ideas
- The Web as a Dynamic Corpus: The paper strongly advocates for the Web's potential as a continuously updated and vast resource for query translation, overcoming the limitations of static dictionaries and corpora.
- Search Result Pages are Rich Sources: Even noisy search result pages, when mined effectively, can yield valuable bilingual information, especially for proper nouns and specialized terms.
- Complementarity of Methods: The chi-square and context-vector methods, while distinct, offer complementary strengths. Combining them leads to more robust and comprehensive translation results.
- Anchor Text vs. Search Results: While anchor texts are useful, mining search result pages offers a more cost-effective and dynamic solution, requiring less upfront crawling and infrastructure.
- Flexibility in Query Specification: The proposed approach allows for more flexible query formulation, as it can handle free text and even translate terms that are not direct equivalents but are semantically related.
Key Results
- High Coverage for Unknown Terms: The search-result-based approach (χ² + CV) demonstrated high inclusion rates for unknown query terms, with over 95% of popular query translations appearing in the top 30-40 snippets and about 70% for random queries.
- Improved CLIR Performance: When used for query translation in a CLIR task, the proposed approach significantly improved Mean Average Precision (MAP) values compared to dictionary-based methods, especially for queries with many unknown terms. The hybrid approach (χ² + CV + AT) achieved the best MAP of 0.279 for K=11.
- Effectiveness Across Query Types: The approach proved effective for translating proper names, technical terms, and even common terms (though with some caveats for highly diverse common terms).
- Cross-Lingual Applicability: Preliminary evaluations suggest the approach is applicable to other language pairs like English-Japanese and English-Korean, achieving reasonable inclusion rates.
Limitations and Future Work
- Dependence on Search Engines: The performance of the approach is inherently tied to the quality and indexing capabilities of the employed search engines.
- Mixed Language Characteristic: The effectiveness of mining search results for translations might be reduced for language pairs that do not exhibit mixed-language characteristics on the Web.
- Common Term Translation: While generally effective, translating common terms with highly diverse meanings can be challenging if their translations don't frequently co-occur with the source term in search results.
- Future Work: The authors suggest further exploring the integration of different similarity estimation methods and leveraging more advanced query expansion and phrase translation techniques to further enhance CLIR performance. They also mention the development of an experimental system called LiveTrans for online query translation.
Summary updated