Hedapen Semantikoa Informazioaren Berreskurapenean

Arantxa Otegi, Eneko Agirre, Xabier ArregiEKAIA 2014 · DOI

TL;DR

This paper tackles the "vocabulary mismatch" problem in Information Retrieval (IR)—the gap created when users and authors use different words for the same concept. By leveraging Natural Language Processing (NLP) techniques like Word Sense Disambiguation (WSD) and semantic relatedness to expand both queries and documents, the authors demonstrate significant improvements in search effectiveness across multiple large-scale datasets.

What Problem Does This Paper Address?

Standard search engines often rely on exact keyword matching. However, human language is messy. The authors highlight two primary hurdles:

  1. Synonymy (Richness): We have many ways to describe the same thing. A user searching for "rabbit" might miss a highly relevant document that only uses the word "hare."
  2. Polysemy (Ambiguity): The same word can mean different things. A query for "tree" intended to find botanical information might be buried under results for "genealogical trees" or "data structures."

These issues lead to the "vocabulary mismatch" problem, where a document is relevant to a query but doesn't share enough exact terms to be ranked highly, or a document is irrelevant but ranked highly because it shares ambiguous keywords.

Methods

The authors propose Semantic Expansion, which involves adding conceptually related terms to either the user's query (Query Expansion) or the documents themselves (Document Expansion).

Key techniques used include:

  • Word Sense Disambiguation (WSD): Instead of treating a word as a simple string of characters, the system uses context to assign a specific "sense" or meaning to it. This allows the system to match meanings rather than just letters.
  • Semantic Relatedness: Moving beyond simple synonyms, the authors use NLP to find "ahaide semantikoak" (semantic relatives). For example, a query about "speedometers" might be expanded with terms like "vehicle," "distance," or "miles per hour."
  • Integration: These semantic features were integrated into the MG4J search engine framework. The authors specifically compared their approach against Pseudo-Relevance Feedback (PRF), a standard IR technique that expands queries based on the top results of an initial search.

Datasets

The study evaluated the proposed methods across three diverse international datasets to ensure robustness:

  1. Robust-WSD: A collection of 166,754 documents from the Cross Language Evaluation Forum (CLEF).
  2. Yahoo!: A subset of 89,610 documents from the Yahoo! Answers platform.
  3. ResPubliQA: A large-scale dataset of 1,379,011 documents used for multilingual question answering tasks.
  4. Basque Context: The system was also adapted for Basque using a Science and Technology corpus provided by the Elhuyar Foundation as part of the Berbatek project.

Key Insights and Ideas

  • Complementary Strengths: One of the most interesting findings is that semantic expansion and PRF are not competitors; they are partners. PRF tends to work best on "easy" queries where the initial results are already somewhat relevant. Semantic expansion, however, shines on "difficult" queries where the initial keyword match is poor.
  • Precision vs. Recall Trade-off: The authors acknowledge that expansion is a double-edged sword. Adding terms increases the chance of finding relevant documents (recall), but if the WSD is inaccurate, it introduces "noise" that can lead to irrelevant results (decreasing precision).
  • The Power of Context: By disambiguating senses before expansion, the system avoids the trap of polysemy. It ensures that a search for a "biological tree" isn't expanded with terms related to "computer science."

Key Results

The experiments yielded consistent improvements in IR effectiveness:

  • Yahoo! Dataset: The authors achieved statistically significant improvements across all metrics for both query and document expansion.
  • Solving Hard Queries: In the ResPubliQA dataset, a specific query regarding "speedometers" saw its Mean Reciprocal Rank (MRR) jump from 0.333 (using standard methods) to 1.0. This means the semantic expansion moved the most relevant document from the third position to the very first.
  • Real-World Application: The research wasn't just theoretical; it was successfully implemented in an online search system for Basque scientific and technical content, proving its viability for languages with fewer resources than English.

Limitations and Future Work

  • Computational Overhead: Techniques like WSD and calculating semantic relatedness are computationally more expensive than simple keyword indexing. Scaling these to real-time, massive web-scale indices remains a challenge.
  • Dependency on Knowledge Bases: The quality of the expansion is heavily dependent on the underlying lexical databases (like WordNet). For languages or specialized domains where these databases are sparse, the benefits may be limited.
  • Evaluation Benchmarks: The authors note that while English has robust IR evaluation environments, there is a lack of similar standardized benchmarks for the Basque language, making it difficult to perform the same level of rigorous quantitative analysis in that context.

Summary updated

Abstract

Informazioaren berreskurapena (IB) erabiltzaile baten informazio-beharra aseko duten dokumentuak bilatzean datza. Beraz, IB sistemak erabiltzaileari laguntza emango dio dokumentu adierazgarriak, alegia, erabiltzaileak behar duen informazioa eduki dezaketen dokumentuak, topatzeko. Horretarako, erabiltzaileak egindako kontsultan oinarritzen gara. Kontsultaren eta dokumentuen arteko parekatze arazoa deiturikoa da IB sistemek aurre egin behar dioten arazo nagusienetako bat: dokumentu bat kontsulta baterako adierazgarria izan daiteke, nahiz eta bietan erabilitako hitzak guztiz berdinak ez izan, eta, alderantziz, dokumentu bat ezadierazgarria izan daiteke kontsulta baterako, nahiz eta termino komun batzuk eduki. Arazo hauek hitzen sinonimiaren eta anbiguotasunaren kausaz gertatzen dira. Lan honetan, kontsulten eta dokumentuen hedapenak egin eta aurre egingo diogu parekatze arazoari, hizkuntzaren prozesamenduko hitzen adiera-desanbiguazioa eta ahaidetasun semantikoa erabiliz. Hiru datu multzotan egindako esperimentu eta analisiek erakusten dute proposatutako hedapen-metodoek parekatze arazoari aurre egiteko balio dutela eta, ondorioz, baita IB sistemaren eraginkortasuna hobetzeko ere.
BibTeX
@inproceedings{otegi2014hedapen,
  author = {Arantxa Otegi and Eneko Agirre and Xabier Arregi},
  title = {{Hedapen Semantikoa Informazioaren Berreskurapenean}},
  booktitle = {EKAIA},
  year = {2014},
  doi = {10.1387/ekaia.11737},
}
Figure