Abstract: Jia-Chen Gu, Zhen-Hua Ling, Xiaodan Zhu, Quan Liu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
2019.
Introduction
Retrieval-based chatbots aim to select the most appropriate response from a candidate pool given a conversation context. However, a persistent challenge in this field is maintaining a consistent personality. This paper, presented at EMNLP-IJCNLP 2019, introduces the Dually Interactive Matching Network (DIM). By treating a bot’s persona as an equal partner to the conversation context, the authors achieved a significant leap in response selection accuracy, outperforming previous state-of-the-art models by over 27%.
The core innovation of DIM lies in its dual-path architecture. Instead of simply appending persona data to the conversation context, DIM matches a candidate response against both the conversation history and the bot's persona profile in parallel. This ensures that the selected reply is not only contextually relevant but also personality-consistent.
The "Personality Crisis" in Chatbots
While modern retrieval-based chatbots are proficient at maintaining context, they often lack a stable identity, leading to inconsistent responses. Previous attempts to address this used persona fusion, where a bot is provided with profile sentences (e.g., "I have two dogs"). These models typically calculated a similarity score between the context and profile to create a "persona-enhanced" context representation.
The authors identify two major flaws in this approach:
- Coarse Granularity: Previous models treated the entire conversation history as a single block, ignoring how different utterances might relate to specific parts of a persona.
- Missing Links: They often ignored the direct interaction between the persona and the candidate response. A response should actively reflect the persona, not just fit the context.
The DIM Architecture
The DIM model builds upon the Interactive Matching Network (IMN) by expanding it into a "dual" structure designed to capture fine-grained interactions.
The architecture consists of five key layers:
- Word Representation Layer: Combines pre-trained GloVe embeddings, task-specific embeddings, and character-level CNNs to handle out-of-vocabulary words.
- Sentence Encoding Layer: Uses BiLSTMs to capture the temporal flow of utterances, profile sentences, and response candidates.
- Matching Layer: The core innovation. The model runs two matching processes in parallel: Context-Response Matching and Persona-Response Matching. It uses a "soft alignment" (attention) mechanism to determine which words in the response relate to specific words in the context or persona.
- Aggregation Layer: Uses another BiLSTM and pooling operations to condense these complex word-level interactions into fixed-length vectors.
- Prediction Layer: A Multi-Layer Perceptron (MLP) takes the final features and outputs a probability score for the candidate.
Experimental Setup
The researchers evaluated DIM using the PERSONA-CHAT dataset, which includes 8,939 training dialogues. Each speaker has a profile of 3-5 sentences. To test robustness, the authors also used a "Revised" version of the dataset where profile sentences are rephrased or generalized, requiring the model to understand semantics rather than relying on simple keyword matching.
Key Results and Insights
The DIM model demonstrated dominant performance across all metrics. On the original "Self Persona" task, DIM achieved a hits@1 of 78.8%, representing a 14.5% improvement over the IMN baseline and a 27.7% improvement over the previous state-of-the-art (FT-PC).
Key insights from the results include:
- Persona as a First-Class Citizen: Formulating the task as a dual matching problem elevates the persona from a supplementary feature to a primary driver of the selection process.
- Utterance-Level Granularity: Matching the persona against individual utterances is superior to matching against the whole context, allowing the model to "remember" relevant persona traits at specific moments.
- Direct Interaction: The direct comparison between the response and the persona is critical. If a persona mentions a love for dogs, the model needs to see the direct link to a response about a "golden retriever" to score it highly.
Conclusion and Future Work
While DIM sets a new benchmark for personalized response selection, limitations remain. The model's performance gains were primarily in "Self Persona" (consistency); gains in "Their Persona" (adapting to the user) were negligible. Future work may focus on better modeling of the partner's persona for more empathetic interactions. Additionally, as a retrieval-based model, DIM is limited by its response pool and the computational complexity of its dual-matching process.
Summary updated