TITA: A Two-stage Interaction and Topic-Aware Text Matching Model

Xingwu Sun, Yanling Cui, Hongyin Tang, Qiuyu Zhu, Fuzheng Zhang, Beihong JinNAACL 2021 · OpenReview

Abstract

In this paper, we focus on the problem of keyword and document matching by considering different relevance levels. In our recommendation system, different people follow different hot keywords with interest. We need to attach documents to each keyword and then distribute the documents to people who follow these keywords. The ideal documents should have the same topic with the keyword, which we call topic-aware relevance. To tackle this problem, we define a three-level relevance in keyword-document matching task: topic-aware relevance, partially-relevance and irrelevance. To capture the relevance between the short keyword and the document at these three levels, we propose a Two-stage Interaction and Topic-Aware text matching model (TITA). TITA introduces a neural topic model to analyze the document's topic and uses a two-stage interaction mechanism to model complex relationships between keywords and documents. Extensive experiments reveal that TITA outperforms well-designed baselines and shows excellent performance in real-world recommendation systems.

Introduction

Modern recommendation systems often rely on matching user-followed "hot keywords" to relevant documents. However, standard text-matching models frequently struggle to distinguish between documents that are truly about a topic and those that merely mention a keyword incidentally. For instance, a user interested in "cherry blossoms" would prefer a travel guide to a festival (topic-aware relevance) over a restaurant review that happens to mention a nearby tree (partial relevance).

To address this "topic gap," we propose TITA. Unlike previous models that focus solely on word-level interactions or independent representations, TITA explicitly models both fine-grained interactions and the latent global topic of the document.

Methodology

The TITA architecture is designed to integrate local semantic features with global topical context through three primary components.

The TITA architecture, comprising a two-stage interaction mechanism (Part A), a neural topic model (Part B), and a joint training mechanism (Part C).

Two-stage Interaction

The first stage utilizes BERT to encode the keyword and document together, capturing initial self-attention signals. The second stage introduces a Cross Interaction Layer, which calculates a similarity matrix between keyword and document embeddings. This allows for independent contextualization, ensuring the model focuses specifically on how the keyword relates to various segments of the document.

Neural Topic Model (NTM)

To capture the "big picture," we implement an Auto-Encoding Variational Bayes (AEVB) NTM. This component processes a bag-of-words representation of the document to produce a latent topic distribution. This helps the model understand the document's overall intent beyond simple lexical overlap.

Joint Training and Memory Hops

Inspired by memory networks, TITA uses a "memory hop" operation to unify these components. The topic-word distributions act as a memory bank that the deep neural representation of the document can "read" from, enriching the final representation with topical information before making a relevance prediction.

Experimental Results

We evaluated TITA using a large-scale Chinese dataset derived from the Meituan recommendation system, consisting of 8,901 keywords and over 66,000 manually labeled keyword-document pairs.

Performance comparison of TITA against various baseline models on the keyword-document matching task.

As shown in the performance table, TITA achieved an accuracy of 75.22%, significantly outperforming standard BERT (69.84%) and other interaction-based models like ARC-II (68.16%). Ablation studies further confirmed that removing either the Neural Topic Model or the second interaction stage led to measurable performance drops, validating the necessity of both components.

Industrial Implementation and Impact

To address the computational complexity of the model in a production environment, we utilize an offline data processor. Relevance scores are pre-calculated and stored in a Key-Value (KV) database, which is then accessed during online retrieval. When deployed in a live recommendation system, TITA increased the click-through rate (CTR) from 6.52% to 10.87%—a substantial 4.35% absolute improvement.

Conclusion

TITA successfully bridges the gap between word-level matching and topic-level understanding. By combining modern deep learning (BERT) with traditional probabilistic concepts (topic modeling) in an end-to-end framework, it provides a high-precision solution for industrial-scale recommendation tasks.

Summary updated

Abstract

In this paper, we focus on the problem of keyword and document matching by considering different relevance levels. In our recommendation system, different people follow different hot keywords with interest. We need to attach documents to each keyword and then distribute the documents to people who follow these keywords. The ideal documents should have the same topic with the keyword, which we call topic-aware relevance. In other words, topic-aware relevance documents are better than partially-relevance ones in this application. However, previous tasks never define topic-aware relevance clearly. To tackle this problem, we define a three-level relevance in keyword-document matching task: topic-aware relevance, partially-relevance and irrelevance. To capture the relevance between the short keyword and the document at above-mentioned three levels, we should not only combine the latent topic of the document with its deep neural representation, but also model complex interactions between the keyword and the document. To this end, we propose a Two-stage Interaction and Topic-Aware text matching model (TITA). In terms of “topic-aware”, we introduce neural topic model to analyze the topic of the document and then use it to further encode the document. In terms of “two-stage interaction”, we propose two successive stages to model complex interactions between the keyword and the document. Extensive experiments reveal that TITA outperforms other well-designed baselines and shows excellent performance in our recommendation system.
BibTeX
@inproceedings{sun2021tita,
  author = {Xingwu Sun and Yanling Cui and Hongyin Tang and Qiuyu Zhu and Fuzheng Zhang and Beihong Jin},
  title = {{TITA: A Two-stage Interaction and Topic-Aware Text Matching Model}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=BeZWYNjAqo},
}
Figure