Modularized Transfomer-based Ranking Framework

Luyu Gao, Zhuyun Dai, Jamie CallanEMNLP 2020 · OpenReview

Abstract: Luyu Gao, Zhuyun Dai, Jamie Callan. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

2020.

Introduction

Transformer-based models like BERT have set new benchmarks in Information Retrieval (IR). However, their deployment in production is often hindered by high computational costs and a "black box" nature. Standard BERT rankers process query-document pairs as a single concatenated sequence, leading to quadratic scaling ($O(N^2)$) and preventing the pre-computation of document representations.

The Modularized REranking System (MORES) addresses these challenges by decoupling text representation from interaction. This modular approach enables offline document processing and provides a window into the model's internal reasoning, achieving up to a 120x speedup on GPUs without sacrificing state-of-the-art accuracy.

The MORES Architecture

MORES decomposes the ranking process into three distinct Transformer-based modules, as illustrated in Figure 1. This separation allows the system to move the most computationally expensive parts of the process offline.

The MORES architecture decouples document and query representation from the interaction layer, allowing for offline document indexing.
  1. Document Representation Module: Embeds document tokens independently of the query. This allows representations to be computed and indexed offline.
  2. Query Representation Module: Embeds query tokens independently.
  3. Interaction Module: The core innovation, using a stack of Interaction Blocks (IB). It performs query-to-document cross-attention (finding matches) followed by query self-attention (aggregating match signals).

To optimize online performance, the authors introduce Reuse-S2, which stores pre-computed "Key" and "Value" linear projections for the cross-attention layer. This strategy significantly reduces real-time computation at the cost of increased storage space.

Efficiency and Performance

Evaluated on major IR benchmarks including MS MARCO, ClueWeb09-B, and Robust04, MORES demonstrates remarkable efficiency. On a GPU, it achieves a 120x speedup over standard BERT rankers for long documents, ranking 1,000 documents in approximately 0.1 seconds. Despite this speed, MORES remains highly effective, performing within a 2% margin of the full BERT ranker's accuracy.

Interpretability and Domain Adaptation

The modular design of MORES offers unique insights into how Transformer rankers function. By visualizing attention maps, the authors discovered a functional division between interaction blocks.

Attention visualizations across modules: (a) Document, (b) Query, (c) 1st Interaction Block, and (d) 2nd Interaction Block.

As shown in Figure 4, the first Interaction Block (c) often focuses on disambiguating query terms. For instance, it identifies that "sc" in a medical context refers to "schizophrenia" by attending to relevant document tokens. Subsequent blocks (d) then focus on matching these refined concepts to the document content.

Furthermore, experiments in domain adaptation suggest that while representations are domain-specific, the interaction mechanism is largely universal. When moving to a new domain, fine-tuning only the representation modules yields performance nearly identical to fine-tuning the entire model, suggesting that the logic of "matching" transfers well across different types of content.

Conclusion

MORES provides a scalable and interpretable framework for neural ranking. While the storage requirements for pre-computed embeddings present a challenge for future optimization, the system's ability to provide real-time, high-accuracy ranking makes it a powerful tool for modern production IR systems.

Summary updated

BibTeX
@inproceedings{gao2020modularized,
  author = {Luyu Gao and Zhuyun Dai and Jamie Callan},
  title = {{Modularized Transfomer-based Ranking Framework}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  url = {https://openreview.net/forum?id=ykSxiR0-A9I},
}
Figure