New Collection Announcement: Focused Retrieval Over the Web

Ivan Habernal, Maria Sukhareva, Fiana Raiber, Anna Shtok, Oren Kurland, Hadar Ronen, Judit Bar-Ilan, Iryna Gurevych

Abstract

Focused retrieval (a.k.a., passage retrieval) is important at its own right and as an intermediate step in question answering systems. We present a new Web-based collection for focused retrieval. The document corpus is the Category A of the ClueWeb12 collection. Forty-nine queries from the educational domain were created. The $100$ documents most highly ranked for each query by a highly effective learning-to-rank method were judged for relevance using crowdsourcing. All sentences in the relevant documents were judged for relevance.
BibTeX
@inproceedings{habernal2016new,
  author = {Ivan Habernal and Maria Sukhareva and Fiana Raiber and Anna Shtok and Oren Kurland and Hadar Ronen and Judit Bar-Ilan and Iryna Gurevych},
  title = {{New Collection Announcement: Focused Retrieval Over the Web}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2016},
  doi = {10.1145/2911451.2914682},
}
Figure