A strong baseline for question relevancy ranking

Ana Valeria González-Garduño, Isabelle Augenstein, Anders Søgaard

Abstract

The best systems at the SemEval-16 and SemEval-17 community question answering shared tasks -- a task that amounts to question relevancy ranking -- involve complex pipelines and manual feature engineering. Despite this, many of these still fail at beating the IR baseline, i.e., the rankings provided by Google's search engine. We present a strong baseline for question relevancy ranking by training a simple multi-task feed forward network on a bag of 14 distance measures for the input question pair. This baseline model, which is fast to train and uses only language-independent features, outperforms the best shared task systems on the task of retrieving relevant previously asked questions.
BibTeX
@inproceedings{gonzalezgarduno2018strong,
  author = {Ana Valeria González-Garduño and Isabelle Augenstein and Anders Søgaard},
  title = {{A strong baseline for question relevancy ranking}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure