TL;DR
This paper introduces a novel neural network architecture that generates customer service responses tailored to a specific personality. By conditioning response generation on target personality traits, the model significantly improves response quality and linguistic variation compared to standard sequence-to-sequence models, as validated by both automated metrics and human evaluation.
What Problem Does This Paper Address?
Automated conversational agents are becoming ubiquitous, and their effectiveness, particularly in customer service, can be significantly enhanced by adapting their persona. While previous work has explored generating responses with a consistent persona, this paper focuses on a more granular aspect: generating responses that explicitly express specific, target personality traits. The challenge lies in encoding these abstract personality traits into a neural generation model and ensuring the generated responses reflect them accurately and naturally, moving beyond generic, consensus-based responses.
Methods
The authors build upon the standard sequence-to-sequence (Seq2Seq) architecture, commonly used for neural response generation. Their key innovation is the introduction of an additional "personality-based features hidden layer." This layer takes a vector representing target personality traits (e.g., from the Big Five model) and learns high-level personality-based features. These learned features are then used to condition the decoder's hidden state at each token generation step, alongside the previous hidden state, the previously generated token, and the attention context. This allows the model to dynamically adjust its response generation based on the desired personality. Unlike prior work that manually defined linguistic features, this approach automatically learns these personality-based features.
Datasets
The primary dataset used is a collection of 87.5K real customer-agent utterance pairs from social media, specifically from customer service Twitter channels. This dataset, originally presented by Xu et al. (2017), covers a wide variety of customer service interactions. For personality trait extraction, the authors utilized the IBM Personality Insights service, which infers traits from social media text.
Key Insights and Ideas
- Personality as a controllable generation parameter: The core idea is treating personality traits not as an emergent property of the data, but as explicit input to guide the generation process.
- Automatic feature learning: The model learns to represent personality traits through a dedicated hidden layer, eliminating the need for manual feature engineering and allowing for more nuanced trait expression.
- Linguistic variation is key: The paper posits that personality traits manifest as stylistic linguistic variations, and conditioning generation on these traits allows for more diverse and appropriate responses.
- Customer service focus: The research specifically targets customer service, where personality traits like conscientiousness and agreeableness have been shown to impact service quality.
Key Results
- Automated Metrics: The proposed PERSONALITY-BASED model achieved significant improvements over a baseline Seq2Seq model. In a "warm start" setting (where agents in test data were also in training data), the model showed improved perplexity and BLEU scores. In a "cold start" setting (where test agents were not in training data), the model demonstrated a relative decrease of 6.7% in perplexity and a 46% relative improvement in BLEU score, highlighting its ability to generalize to new personality distributions.
- Human Evaluation: A human evaluation on 60 customer utterances showed that responses generated by the PERSONALITY-BASED model were judged as more expressive of target personality traits (agreeableness and conscientiousness) in 61% of cases compared to low-trait versions.
Limitations and Future Work
- Limitations: The paper acknowledges that the PERSONA-BASED model (Li et al., 2016) achieved similar perplexity but higher BLEU in the warm start setting, attributing this to its ability to learn agent-specific personas beyond just personality traits. A key limitation of the PERSONA-BASED model is its inability to generate content for agents not present in the training data.
- Future Work: The authors suggest extending the model to generate responses adapted to the customer's personality traits, as well as applying the model to other domains like education systems.
Summary updated