Having Your Cake and Eating it Too: Training Neural Retrieval for Language Inference without Losing Lexical Match
Lists
Only you see this
Abstract
We present a study on the importance of information retrieval (IR) techniques for both the interpretability and the performance of neural question answering (QA) methods. We show that the current state-of-the-art transformer methods (like RoBERTa) encode poorly simple information retrieval (IR) concepts such as lexical overlap between query and the document. To mitigate this limitation, we introduce a supervised RoBERTa QA method that is trained to mimic the behavior of BM25 and the soft-matching idea behind embedding-based alignment methods. We show that fusing the simple lexical-matching IR concepts in transformer techniques results in improvement a) of their (lexical-matching) interpretability, b) retrieval performance, and c) the QA performance on two multi-hop QA datasets. We further highlight the lexical-chasm gap bridging capabilities of transformer methods by analyzing the attention distributions of the supervised RoBERTa classifier over the context versus lexically-matched token pairs.
BibTeX
@inproceedings{yadav2020having,
author = {Vikas Yadav and Steven Bethard and Mihai Surdeanu},
title = {{Having Your Cake and Eating it Too: Training Neural Retrieval for Language Inference without Losing Lexical Match}},
booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
year = {2020},
doi = {10.1145/3397271.3401311},
}Loading related content...