Are we on the Right Track?: An Examination of Information Retrieval Methodologies

Enrique Amigó, Hui Fang, Stefano Mizzaro, ChengXiang Zhai

Abstract

The unpredictability of user behavior and the need for effectiveness make it difficult to define a suitable research methodology for Information Retrieval (IR). In order to tackle this challenge, we categorize existing IR methodologies along two dimensions: (1) empirical vs. theoretical, and (2) top-down vs. bottom-up. The strengths and drawbacks of the resulting categories are characterized according to 6 desirable aspects. The analysis suggests that different methodologies are complementary and therefore, equally necessary. The categorization of the 167 full papers published in the last SIGIR (2016 and 2017) and ICTIR (2017) conferences suggest that most of existing work is empirical bottom-up, suggesting lack of some desirable aspects. With the hope of improving IR research practice, we propose a general methodology for IR that integrates the strengths of existing research methods.
BibTeX
@inproceedings{amigo2018are,
  author = {Enrique Amigó and Hui Fang and Stefano Mizzaro and ChengXiang Zhai},
  title = {{Are we on the Right Track?: An Examination of Information Retrieval Methodologies}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2018},
  doi = {10.1145/3209978.3210131},
}

Loading related content...

Figure