Less is more: Eliminating index terms from subordinate clauses

Simon Corston-Oliver, William B. DolanACL 1999 · DOI

In the field of Information Retrieval (IR), the prevailing philosophy has often been that more data leads to better results. Researchers frequently attempt to enhance search engine performance by adding complex linguistic features, such as noun phrases or semantic relations, to document indexes. However, research from Microsoft Research challenges this "more is better" paradigm. By leveraging Natural Language Processing (NLP) to identify and remove linguistically peripheral information, the authors demonstrate that search indexes can be reduced by approximately 30% with negligible impact on retrieval quality.

The "Aboutness" Problem and Index Efficiency

The core challenge in IR is determining a document's "aboutness"—identifying the primary topics that make a document relevant to a query. Traditional indexing treats all terms with equal weight, leading to massive index sizes as the web grows. The authors argue that terms appearing only in subordinate clauses (e.g., "which was wearing a green hat") are poor indicators of a document's central theme compared to terms in main clauses.

To address this, the researchers employed a "subtractive" approach. Using the Microsoft English Grammar (MEG) parser, they analyzed the syntactic structure of documents to identify specific subordinate structures, such as relative, adverbial, and complement clauses. They then tested a Natural Language Matching (NLM) index that omitted terms found exclusively within these subordinate structures.

Experimental Results: Web and Encyclopedia Corpora

The study evaluated this approach on two distinct datasets: a Web Sample of 3,440 documents and the Encarta 99 electronic encyclopedia. The results across both corpora were remarkably consistent regarding storage efficiency, achieving an index size reduction of 29% to 31.4%.

Performance comparison of Baseline NLM vs. Subordinate Clause removal on the Web Sample.

As shown in Figure 1, the performance hit on the Web Sample was minimal, with the F-measure (a harmonic mean of precision and recall) dropping by only 0.82%. In contrast, a "dumb" pruning method involving random term deletion was found to be over five times more damaging to search performance than this targeted linguistic pruning, even when the random method removed fewer terms.

Linguistic Insights and Genre Differences

The effectiveness of this pruning varies by the type of subordinate clause and the genre of the text. Applying Rhetorical Structure Theory (RST), the authors mapped main clauses to informational "nuclei" and subordinate clauses to "satellites."

Impact of removing specific types of subordinate clauses on search performance for the Web Sample.

Analysis of specific clause types (Figure 5) revealed that removing relative clauses (RELCL) actually improved the F-measure by 0.33% on the web. This suggests that relative clauses often contain "noise" that can lead to false positives. However, the impact was more pronounced in the Encarta dataset (a 4.91% drop in F-measure), likely because encyclopedic entries often use relative clauses to provide essential definitional details.

Conclusion

This research proves that linguistic "satellites" can often be safely ignored in the indexing process. While the current study used a binary deletion method, future work may explore weighting terms based on their grammatical role. By recognizing that not all clauses are created equal, search engines can achieve significant storage savings while maintaining high levels of precision and recall.

Summary updated

BibTeX
@inproceedings{corstonoliver1999less,
  author = {Simon Corston-Oliver and William B. Dolan},
  title = {{Less is more: Eliminating index terms from subordinate clauses}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1999},
  doi = {10.3115/1034678.1034734},
}
Figure