TL;DR
Traditional search engines are highly efficient but often miss the underlying meaning of user queries because they rely heavily on exact keyword matching. This paper introduces a modular system that intercepts the top results from major search engines (Google, Yahoo, Bing) and re-ranks them using the WordNet ontology to account for semantic meaning (synonyms and broader concepts), resulting in a noticeable improvement in search relevance, especially for complex queries.
What Problem Does This Paper Address?
Information Retrieval (IR) systems, including massive web search engines, are incredibly efficient at finding documents that contain specific keywords. However, they often struggle with semantics—the actual meaning and intent behind the user's query and the vocabulary used in the documents. For example, if a user searches for "automobile," a purely keyword-based system might rank a highly relevant document lower simply because it uses the word "car" instead.
The authors tackle the challenge of improving the selectivity and evaluation of existing search tools. They aim to inject a layer of semantic understanding into the ranking process, ensuring that documents conceptually related to the query are prioritized even if exact keyword matches are absent.
Methods
The authors propose a modular architecture that acts as a semantic wrapper around existing search engines. The pipeline consists of several distinct steps:
- Search & Extraction: The system passes the user's query to Google, Yahoo, and Bing, retrieving the top 20 HTML results. It parses these pages to extract the title, abstract, and URL, stripping away useless HTML tags.
- Semantic Projection (The Novelty): Instead of just using the raw query words, the system projects the query onto the WordNet 3.0 ontology. It extracts synonyms and hypernyms (broader, overarching concepts) to build an expanded "semantic vector."
- Calculation: The system employs the Vector Space Model (VSM). It calculates term weights using a modified TF-IDF formula (which accounts for document length) and then measures the Cosine Similarity between the expanded semantic query vector and the extracted document vectors.
- Re-ranking: Finally, the system re-orders the original top 20 results based on these new semantic similarity scores, presenting a conceptually refined list to the user.
Datasets
The system does not use a static training dataset; it operates dynamically on live web data. However, it relies on two main resources for its operation and evaluation:
- Linguistic Resource: WordNet 3.0 (containing over 155,000 words and 117,000 concepts) is used as the foundational ontology for semantic projection.
- Evaluation Corpus: The authors generated a custom evaluation set by running 25 specific search scenarios (15 "simple" everyday queries and 10 "complex" specialized/rare-word queries) through the search engines, resulting in a pool of 500 web pages evaluated via a manual scoring grid.
Key Insights and Ideas
- Semantics rescue complex queries: While standard search engines do well on simple, everyday queries, their performance drops on complex or specialized searches. Injecting semantic understanding provides the most significant relevance boost precisely where traditional engines struggle.
- Ontologies can be bolted onto existing systems: You don't need to build a multi-billion-dollar web crawler to get semantic search. By using a meta-search approach that re-ranks the top results of existing engines, you get the best of both worlds: the massive indexing power of Google/Yahoo and the nuanced understanding of a linguistic ontology.
- The double-edged sword of synonyms: Expanding queries with synonyms improves overall relevance but can slightly increase the rate of "redundant results." Because the broadened vocabulary matches more pages, the system is slightly more likely to pull in multiple different pages from the exact same website.
Key Results
- Overall Improvement: Semantic ranking improved the overall average effectiveness score for both Google (from 7.62 to 8.29 out of 10) and Yahoo (from 6.93 to 7.02).
- Massive Gains in Relevance: When looking strictly at the "results relevance" criterion, Yahoo saw a massive jump from 5.06 (classical ranking) to 7.66 (semantic ranking). Google improved from 5.72 to 6.12.
- Complex vs. Simple: The performance gap between classical and semantic ranking widened for complex queries. For Google, semantic ranking added 0.67 points for simple queries, but 0.75 points for complex ones.
- Link Quality: Semantic ranking showed slight improvements in filtering out "parasite" (spam/promotional) pages and handling dead links compared to the default search engine rankings.
Limitations and Future Work
- Processing Overhead: The authors explicitly note that the system currently only evaluates the top 20 results. Expanding this to cover all returned results would logically increase processing time, highlighting a fundamental trade-off between semantic depth and computational speed.
- Redundancy: As noted in the results, the semantic expansion slightly worsened the "non-redundant results" score compared to classical ranking.
- Future Directions: While not detailed in a dedicated section, the authors emphasize the highly modular nature of their architecture. This implies that future work could easily swap the general-purpose WordNet for domain-specific ontologies (e.g., medical, legal, or geographic) to create highly specialized, highly accurate semantic search wrappers for specific industries.
Summary updated