Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention

Hsuan Su, Jiun-Hao Jhan, Fan-Yun Sun, Saurav Sahay, Hung-yi Lee

Abstract

Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into what really separates humans from chatbots -- humans intrinsically understand the effect their responses have on the interlocutor and often respond with an intention such as proposing an optimistic view to make the interlocutor feel better. This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework includes a guiding chatbot and an interlocutor model that plays the role of humans. The guiding chatbot is assigned an intention and learns to induce the interlocutor to reply with responses matching the intention, for example, long responses, joyful responses, responses with specific words, etc. We examined our framework using three experimental setups and evaluated the guiding chatbot with four different metrics to demonstrate flexibility and performance advantages. Additionally, we performed trials with human interlocutors to substantiate the guiding chatbot's effectiveness in influencing the responses of humans to a certain extent. Code will be made available to the public.

Introduction: The Missing Element of Intent

Current research in conversational AI primarily focuses on response fluency and coherence. While these models can mimic human speech patterns, they often lack the underlying intent that characterizes human communication. When humans interact, we frequently aim for specific outcomes—cheering up a friend, eliciting detailed information, or steering a topic. Standard chatbots are passive, responding to prompts without considering the long-term impact on the listener.

Conceptual comparison between a typical passive chatbot and the proposed guiding chatbot that responds with specific intentions.

This research introduces a framework designed to bridge this gap by training chatbots to "understand" the impact of their utterances and actively guide conversations toward desired interlocutor responses.

Methodology: The Guiding Chatbot Framework

The proposed framework utilizes a Reinforcement Learning (RL) setup involving two distinct models:

  1. The Guiding Chatbot ($C$): Based on DialoGPT, this model is trained to achieve a specific "controllable factor" or intention.
  2. The Interlocutor Model ($I$): A fixed model that simulates a human listener, providing the feedback necessary for the guiding chatbot to learn.
The proposed framework architecture showing the interaction between the Guiding Chatbot, the Interlocutor Model, and the dual-reward system.

The training process is driven by a dual-reward system:

  • Intention Reward ($R_I$): This reward is calculated based on the interlocutor's response. For instance, if the goal is to induce joy, the reward increases if the interlocutor's subsequent response is detected as joyful.
  • Coherence Reward ($R_C$): To ensure the chatbot remains relevant and fluent, a pre-trained GPT-2 model evaluates the probability of the chatbot's output. This prevents the model from "hacking" the reward system with nonsensical but effective triggers.

Experimental Setup and Datasets

The researchers evaluated the framework across three primary intentions:

  • Sentence Length: Encouraging the listener to provide more verbose responses.
  • Emotion: Inducing specific emotional states (e.g., joy, anxiety) using a BERT-based emotion detector.
  • Specific Words: Nudging the listener toward categories like Food, Sports, or "Bad Words."

The models were fine-tuned using the EmpatheticDialogues (ED) dataset, which contains 25k conversations grounded in emotional contexts. Additionally, the Cornell Movie Dialogues dataset was used to analyze the inherent biases of different interlocutor models, such as the tendency of certain bots to use aggressive language.

Key Results

The framework demonstrated significant success across all tested categories:

  • Length Induction: The guiding chatbot successfully increased average interlocutor response length from 3 to 10 words.
  • Emotional Guidance: The model effectively minimized the Mean Square Error (MSE) between the target emotion and the interlocutor's actual response emotion.
  • Topic Steering: In the "Food" category, the chatbot significantly increased the frequency of food-related terms in interlocutor replies.
  • Human Validation: Trials with 19 human participants confirmed that the chatbot could influence real people to speak with more anxiety while maintaining a reasonable level of conversational relevance (scoring ~3.21/5).

Discussion and Ethical Considerations

While effective, the model occasionally relied on repetitive strategies, such as repeatedly asking about a favorite food to steer the topic. Furthermore, a trade-off exists between achieving an intention and maintaining high conversational relevance; as the intention reward increases, human-perceived relevance tends to decrease.

The authors highlight significant ethical risks, noting that such technology could be misused for emotional manipulation or social antagonism. They advocate for the development of "Intention Classifiers" and transparent labeling of AI-generated intent to mitigate these risks.

Summary updated

Abstract

Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into what really separates humans from chatbots -- humans intrinsically understand the effect their responses have on the interlocutor and often respond with an intention such as proposing an optimistic view to make the interlocutor feel better. This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework includes a guiding chatbot and an interlocutor model that plays the role of humans. The guiding chatbot is assigned an intention and learns to induce the interlocutor to reply with responses matching the intention, for example, long responses, joyful responses, responses with specific words, etc. We examined our framework using three experimental setups and evaluated the guiding chatbot with four different metrics to demonstrate flexibility and performance advantages. Additionally, we performed trials with human interlocutors to substantiate the guiding chatbot's effectiveness in influencing the responses of humans to a certain extent. Code will be made available to the public.
BibTeX
@inproceedings{su2021put,
  author = {Hsuan Su and Jiun-Hao Jhan and Fan-Yun Sun and Saurav Sahay and Hung-yi Lee},
  title = {{Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=H7ArhsUAev},
}
Figure