TL;DR
Web image search engines struggle with ambiguous queries, returning irrelevant results due to polysemy (words with multiple meanings). This paper introduces IDIOM, a system that improves image search relevance by automatically selecting the best translations of a query's intended sense across multiple languages, outperforming Google Images and other baseline methods.
What Problem Does This Paper Address?
When we search for images online, we often use words that have multiple meanings (polysemy). For example, "spring" can refer to the season, a water source, a coiled spring, or the act of jumping. Current image search engines, like Google Images, often merge results from all these meanings, leading to a mix of irrelevant images. This is particularly problematic for subjective searches where users have a specific meaning in mind. Simple query modifications, like adding related words, don't always solve the problem because the search engine might not associate those added words with the correct images. The paper aims to tackle this challenge by finding a way to search for images that are relevant to a specific sense of a word, not just the word itself.
Methods
The core of IDIOM's approach lies in its ability to select the most effective translations for a given sense. The system leverages PANDICTIONARY, a sense-disambiguated, multilingual dictionary that provides multiple translations for each word sense, along with a probability score indicating the confidence in that translation.
The process involves three main steps:
- Sense Selection: The user is presented with the different senses of their query word and selects the intended one.
- Translation Shortlisting: This is the key innovation. IDIOM uses a learned classifier to predict which translations of the selected sense are likely to yield high-quality, relevant images. The classifier considers features such as:
- Language Coverage: Is the translation in a widely spoken language (e.g., English, French, German)?
- Monosemy: Does the translation primarily refer to the intended sense, or is it also ambiguous?
- Homography: Does the translation have many homographs in other languages?
- Sense Popularity and Order: Is the intended sense the dominant or most common meaning for that word in the target language?
-
Part of Speech, Abstractness/Concreteness: These linguistic features also inform the prediction. The classifier ranks translations based on a confidence score, and the top three translations (above a minimum confidence threshold) are selected.
-
Image Search: IDIOM queries Google Images using these three shortlisted translations and presents the combined results to the user.
Datasets
- PANDICTIONARY: Used to acquire senses and translations. The paper states PANDICTIONARY has about 80,000 senses and 1.8 million translations at 0.9 precision.
- Hand-labeled Dataset: A dataset of 767 random word-sense pairs was created for training the classifier. Pairs were labeled as "positive" if searching the word returned at least one good image for that sense in the top three results.
- Evaluation Queries: 50 randomly selected English queries from PANDICTIONARY were used for evaluation, resulting in 134 senses. These queries included short phrases and words with varying degrees of polysemy.
Key Insights and Ideas
- Translation is Key: The paper strongly suggests that querying with well-chosen translations of a specific sense can significantly improve image search relevance compared to using the original, potentially ambiguous, source word.
- The Power of a Good Classifier: Simply using random translations (as in PANIMAGES) is not as effective as using a learned classifier to pick the "good" ones. The classifier's ability to predict translation quality is crucial.
- Ambiguity is the Enemy: The performance gap between IDIOM and other methods widens considerably for highly polysemous queries, highlighting the severity of the problem IDIOM addresses.
- English Isn't Always Best: While English is favored, the classifier intelligently chooses less ambiguous translations in other languages (like French for "coil" instead of "spring") when the English word is highly ambiguous.
Key Results
- IDIOM outperforms Google Images and other competing algorithms, returning 22% more relevant images overall.
- IDIOM returns at least one good image for 35% more senses than its closest competitor.
- The learned classifier achieved 69% classification accuracy on the hand-labeled dataset, outperforming Naive Bayes (63%) and SVMs (62%).
- IDIOM's performance advantage is most pronounced for highly polysemous queries.
- IDIOM consistently produces better results across different parts of speech (nouns, verbs, adjectives) and abstract vs. concrete senses.
Limitations and Future Work
- User Burden: While IDIOM removes the burden of choosing translations from the user, it still requires the user to select the correct sense of the query word.
- Future Work: The authors plan to release an IDIOM website. They also aim to incorporate knowledge from WordNet and cross-lingual links in Wikipedia to expand IDIOM's coverage beyond the senses available in PANDICTIONARY.
Summary updated