Abstract
Question Answering (QA) systems are used to provide proper responses to users' questions automatically. Sentence matching is an essential task in the QA systems and is usually reformulated as a Paraphrase Identification (PI) problem. Given a question, the aim of the task is to find the most similar question from a QA knowledge base. In this paper, we propose a Multi-task Sentence Encoding Model (MSEM) for the PI problem, wherein a connected graph is employed to depict the relation between sentences, and a multi-task learning model is applied to address both the sentence matching and sentence intent classification problem. In addition, we implement a general semantic retrieval framework that combines our proposed model and the Approximate Nearest Neighbor (ANN) technology, which enables us to find the most similar question from all available candidates very quickly during online serving. The experiments show the superiority of our proposed method as compared with the existing sentence matching models.
Introduction
Modern Question Answering (QA) systems, such as virtual assistants, rely heavily on their ability to retrieve semantically similar questions from vast knowledge bases. Traditionally, this has been approached as a two-stage process: a coarse keyword-based retrieval followed by a computationally intensive re-ranking step. However, this architecture often suffers from a "re-ranking bottleneck" where the initial retrieval phase fails to capture semantically related but lexically different candidates.
To address these challenges, researchers at Baidu have introduced the Multi-task Sentence Encoding Model (MSEM). This model enhances semantic retrieval by simultaneously learning paraphrase identification and sentence intent classification, achieving a significant performance boost in real-world deployments.
The Traditional Retrieval Bottleneck
Most Information Retrieval (IR)-based QA systems follow a linear pipeline. As shown in Figure 1, the process begins with question processing and retrieval of top-K candidates using term-based methods (like BM25 or Lucene). These candidates are then re-ranked using complex similarity computations.
This approach has two primary flaws:
- Recall Limitations: If the keyword search misses the correct question due to vocabulary mismatch, the re-ranker never sees it.
- Independence Assumption: Traditional models treat sentence pairs as independent, ignoring the transitive nature of paraphrasing (if A matches B and B matches C, then A matches C).
The MSEM Framework
MSEM shifts the paradigm from interaction-based re-ranking to an encoding-based retrieval framework.
Multi-task Learning and Intent Classification
The core innovation of MSEM is its multi-task learning objective. Beyond simple binary paraphrase identification, the model is trained to classify the "intent" of a sentence. The authors construct a connected graph where nodes are questions and edges represent paraphrase relations. As illustrated in Figure 6, these connected components form clusters, each representing a distinct semantic intent.
By training the model to identify which cluster a sentence belongs to, it learns a more robust semantic representation that captures the underlying meaning rather than just surface-level word overlap.
Sentence Encoder Architecture
The MSEM encoder transforms sentences into fixed-length vectors using three key components:
- BiGRU: Captures sequential dependencies and context.
- Attention Recurrent Unit (ARU): A novel layer utilizing position-wise multi-head attention. Unlike standard RNNs, the ARU can be computed in parallel, significantly increasing training and inference speed on GPUs.
- Attentive Pooling: Instead of simple averaging, this mechanism learns to weight the most informative words when generating the final sentence embedding.
Semantic Retrieval with ANN
By using an encoding-based model, MSEM allows for the pre-computation of vectors for the entire knowledge base. During online serving, the system uses Approximate Nearest Neighbor (ANN) search (via the Annoy library) to query the entire database in milliseconds. This bypasses the limitations of keyword-based retrieval entirely.
Experimental Results
The model was rigorously tested on several datasets, including Quora Question Pairs, LCQMC (Chinese), and real-world customer service logs (TCS).
- State-of-the-Art Accuracy: MSEM achieved 88.86% accuracy on the Quora dataset, outperforming established models like BiMPM.
- Real-world Impact: In a telephone customer service deployment, the MSEM framework delivered an F1-score of 84.87%, representing a 14.26% improvement over the baseline keyword-based system.
- Ablation Study: Removing the multi-task intent classification task resulted in a 0.75% drop in accuracy, confirming that intent-based regularization is crucial for semantic depth.
Conclusion
The Multi-task Sentence Encoding Model (MSEM) demonstrates that combining multi-task learning with efficient encoding architectures can bridge the gap between accuracy and speed in large-scale QA systems. By leveraging intent-based graph structures and ANN retrieval, MSEM provides a scalable solution for high-performance semantic search in real-world applications.
Summary updated