Promoting ranking diversity for genomics search with relevance-novelty combined model

Xiaoshi Yin, Zhoujun Li, Jimmy Huang, Xiaohua HuBMC Bioinformatics 2011 · OpenReview

Abstract

In the biomedical domain, queries often seek a comprehensive list of entities—such as genes, proteins, or diseases—covering various aspects of a topic. Traditional retrieval models, which assume document relevance is independent, often produce redundant results. This paper introduces RelNov, a relevance-novelty combined model based on an undirected graphical framework. RelNov incorporates two components: an aspect-term relevance model and an aspect-term novelty model. By leveraging Wikipedia for aspect detection, the model achieves significant performance gains, including a 16.4% improvement in Aspect MAP on the TREC 2007 Genomics track. Our results demonstrate that integrating aspects and document novelty effectively promotes ranking diversity and improves overall retrieval relevance.

Introduction

Biologists frequently search for information that requires a diverse set of answers. For instance, a query about proteins associated with a specific disease should ideally return a list of different relevant proteins rather than multiple documents discussing the same one. However, standard Information Retrieval (IR) systems typically rank documents based on individual relevance scores, ignoring the redundancy between results. This lack of "ranking diversity" is particularly problematic in genomics, where complex terminology and synonyms can mask repetitive information.

Methodology: The RelNov Model

To address the redundancy problem, we propose the RelNov model. Unlike traditional models, RelNov treats the ranking process as a search for documents that are both relevant to the query and novel relative to previously seen results.

Graphical Framework

The model is built on an undirected graphical framework (specifically a Markov Random Field). It defines potential functions for aspects, terms, topic relevance, and document novelty to calculate the probability of a document being both useful and unique.

The general framework of the RelNov model as an undirected graphical model, illustrating the relationships between document parameters, background models, relevance, and novelty.

Aspect Detection via Wikipedia

A key innovation is the use of Wikipedia as an external knowledge base to identify "aspects"—semantic concepts like specific biological entities. By "Wikifying" documents, the system maps various phrases (e.g., "SLE" and "Systemic lupus erythematosus") to a single canonical concept. This allows the model to recognize redundancy even when different terminology is used.

Two-Tiered Modeling

RelNov evaluates documents using both literal terms and underlying aspects. This dual approach ensures that the system captures both the specific nuances of the text and the high-level information "nuggets" provided by the document.

Detailed structures of the Term-Aspect Relevance Model (a) and the Term-Aspect Novelty Model (b), showing how terms and aspects contribute to the final scoring.

Experimental Evaluation

The model was tested using the TREC 2006 and 2007 Genomics track collections, which include over 160,000 full-text biomedical documents. Performance was measured using Aspect MAP (Mean Average Precision), a metric designed to reward the discovery of diverse information aspects.

Key Results

  • Performance Gains: RelNov achieved a 16.4% improvement in Aspect MAP over the top-performing system in the TREC 2007 Genomics track and a 9.8% improvement in passage-level retrieval.
  • Robustness: The model improved performance across various baselines, including standard Okapi BM25 and more advanced systems like NLMinter.
  • Knowledge Base Efficacy: Wikipedia was found to be more effective for aspect detection than specialized biomedical ontologies like UMLS, largely due to its superior context for disambiguating acronyms.

Conclusion

The RelNov model demonstrates that explicitly modeling document novelty and semantic aspects significantly enhances the quality of genomics search results. By moving beyond keyword matching to concept-based diversity, the system provides biologists with more comprehensive and less redundant information. Future work will focus on improving the efficiency of the novelty calculations and exploring deeper semantic relationships between biological entities.

Summary updated

Abstract

Background In the biomedical domain, the desired information of a question (query) asked by biologists usually is a list of a certain type of entities covering different aspects that are related to the question, such as genes, proteins, diseases, mutations, etc. Hence it is important for a biomedical information retrieval system to be able to provide comprehensive and diverse answers to fulfill biologists’ information needs. However, traditional retrieval models assume that the relevance of a document is independent of the relevance of other documents. This assumption may result in high redundancy and low diversity in the retrieval ranked lists. Results In this paper, we propose a relevance-novelty combined model, named RelNov model, based on the framework of an undirected graphical model. It consists of two component models, namely the aspect-term relevance model and the aspect-term novelty model. They model the relevance of a document and the novelty of a document respectively. We show that our approach can achieve 16. 4% improvement over the highest aspect level MAP reported in the TREC 2007 Genomics track, and 9. 8% improvement over the highest passage level MAP reported in the TREC 2007 Genomics track. Conclusions The proposed combination model which models aspects, terms, topic relevance and document novelty as potential functions is demonstrated to be effective in promoting ranking diversity as well as in improving relevance of ranked lists for genomics search. We also show that the use of aspect plays an important role in the model. Moreover, the proposed model can integrate various different relevance and novelty measures easily.
BibTeX
@article{yin2011promoting,
  author = {Xiaoshi Yin and Zhoujun Li and Jimmy Huang and Xiaohua Hu},
  title = {{Promoting ranking diversity for genomics search with relevance-novelty combined model}},
  journal = {BMC Bioinformatics},
  year = {2011},
  url = {https://openreview.net/forum?id=UUWXURLuM0X},
}
Figure