TL;DR
This paper introduces a novel "Context Specific Similarity Expansion" (CSSE) technique that automatically builds a context-aware web corpus to discover semantic relationships between terms. By leveraging this external corpus, CSSE significantly improves the accuracy of document similarity computations, outperforming existing methods by up to 50%.
What Problem Does This Paper Address?
Modern information retrieval tasks like document retrieval, clustering, and summarization heavily rely on accurately computing the similarity between text documents. The standard approach, the vector space model, often struggles with the "vocabulary problem" – the fact that people use different words to describe semantically similar concepts. For instance, "car" and "automobile" are treated as distinct terms, leading to inaccurate similarity scores. While previous attempts to address this by mining term co-occurrences have shown mixed results, this paper aims to improve similarity computation by discovering more robust semantic relationships between terms, particularly by leveraging external web data.
Methods
The core of the paper is the Context Specific Similarity Expansion (CSSE) technique. It involves:
- Building a Web Corpus: A heuristic algorithm is used to query commercial search engines (like AltaVista) with terms from a target collection. "Context hints" – top terms from the target collection – are added to queries to ensure the harvested web pages are contextually relevant.
- Semantic Similarity Discovery: Co-occurrence analysis is performed on the built web corpus. A signal-to-noise ratio formula (log N * df(t1,t2) / (df(t1) * df(t2))) is used to compute the similarity between term pairs. A threshold parameter (
Thresh) is applied to filter out weak associations. - Vector Expansion: The discovered term-to-term similarity matrix is used to expand the document vectors of the target collection. This involves adding related terms to a document's vector with weights proportional to their similarity and global inverse document frequency (IDF). An adjustment factor (
a) controls the degree of expansion.
The novelty lies in using an external, contextually relevant web corpus for mining, rather than relying solely on the target collection ("self-mining").
Datasets
- Reuters Collection: Used as the target collection for experiments. This collection is described as a standard test collection for text categorization. The authors selected 78 "commodity code" topics, resulting in 1841 documents after filtering.
- World Wide Web: Used as the external corpus for mining semantic relationships. Approximately 19,198 web pages were harvested and mined.
Key Insights and Ideas
- External Corpus is Crucial: Mining semantic relationships from an external, contextually relevant web corpus (like the WWW) is significantly more effective than "self-mining" (mining only within the target collection). This is because external corpora offer a richer source of co-occurrence data, leading to more reliable associations.
- Context Matters: Including "context hints" from the target collection when building the external corpus dramatically improves results. This suggests that tailoring the mining corpus to the specific domain of interest is key.
- Unimodal Performance Curves: The error reduction curves as a function of average document vector change (
Ca) and threshold (Thresh) are generally unimodal. This makes parameter tuning more manageable, as there's a clear peak to aim for. - Similarity Computation is General: The paper emphasizes that improving similarity computation is a fundamental task that benefits a wide range of downstream applications, not just document retrieval.
Key Results
- Significant Error Reduction: CSSE reduced similarity errors by up to 50%, which is twice the improvement seen with other known techniques like Latent Semantic Indexing (LSI) and Pseudo Relevance Feedback (PRF) within the same experimental framework.
- Parameter Sensitivity: The technique's performance is sensitive to the choice of parameters, specifically the similarity threshold (
Thresh) and the adjustment factor (a). However, the authors suggest safe ranges for these parameters (Threshin .2-.4 andCain .1-.5) that offer significant improvements. - Comparison to Other Techniques:
- LSI: Showed negative error reduction on the entire Reuters collection but was effective on "tougher" topics when the number of semantic axes was comparable to the number of topics.
- PRF: Achieved a maximum improvement of around 21%, comparable to "self-mining" but less than CSSE. The authors note that CSSE can be used in addition to PRF.
- Efficiency: The expansion process for the entire Reuters collection took less than a minute on a workstation, making it suitable for online processing.
Limitations and Future Work
- Parameter Tuning: The sensitivity of the results to parameters like
ThreshandCais acknowledged as a limitation, similar to other IR techniques. - Single Collection: The experiments were conducted on a single collection (Reuters). The authors hope their results generalize but suggest studying other collections.
- Future Work:
- Overcoming limitations by using more than a single collection.
- Studying other experimental setups, such as document retrieval, text categorization, and topic detection.
- Investigating the impact of using larger external corpora.
Summary updated