Abstract
Ongoing chatting is an important step for conversational agents to build long-term connections with people. However, people tend to quickly lose interest in chatting if the conversational agent’s words are not engaging enough. In this paper, we present a novel task of increasing users’ willingness to continue talking to the agent. We collect a dataset named ContinuousChat by: (i) collecting personas and revising them, and then expanding the personas to detailed-personas through experiences, daily life, future plans, or interesting stories; (ii) expanding detailed-personas into the dialogues, and inject emotions and feelings into them; (iii) rewriting the dialogues in specific styles through few-shot prompt, conditioning on handwritten style-specific examples. We benchmark LLMs on ContinuousChat Dataset using both fine-tuning and in-context learning settings. Experiments over publicly available models demonstrate that although there is substantial room for improvement in generating style-specific dialogues, our ContinuousChat dataset is valuable in guiding conversational agents to generate more attractive dialogues and increase users’ willingness to continue the conversations.
Introduction
Most open-domain dialogue research suffers from the "Two Strangers" trap. Existing datasets like PersonaChat usually feature two agents meeting for the first time, exchanging basic facts (e.g., "I have a dog," "I like pizza"). Once these basic facts are exhausted, the conversation often dies, and users lose interest. To build long-term connections, conversational agents must move beyond simple fact exchange toward more engaging, continuous interaction.
The authors identify three critical missing components in current models: depth (detailed memories and routines), emotion (reflecting feelings rather than just facts), and style (adopting specific linguistic "vibes"). The core challenge is increasing Attraction—a metric representing how likely a user is to continue interacting with the agent rather than dropping off out of boredom.
The ContinuousChat Dataset
To address these gaps, the researchers developed CONTINUOUSCHAT, a new Chinese dialogue dataset containing 25,000 dialogues and over 300,000 utterances. The dataset creation follows a three-stage pipeline:
- Persona Expansion: Starting with base personas from the DuLeMon dataset, the authors used ChatGPT to expand them into "Detailed Personas." Instead of a simple fact like "I like tea," the persona is enriched with context, such as drinking tea on a balcony every morning to feel relaxed.
- Base Dialogue Generation: These detailed personas are transformed into multi-turn dialogues infused with specific emotions. Each sentence is marked with an emotion label (e.g., "happy," "sentimental") to guide the model's tone.
- Stylistic Rewriting: To maintain variety and engagement, the base dialogues were rewritten into 13 distinct linguistic styles using few-shot prompting.
Stylistic Diversity
The 13 styles range from common tones like "Lively" and "Humorous" to culturally specific Chinese styles such as "Martial-arts novel," "Chinese-Internet slang," and "Cross-talk" (Xiangsheng). This variety ensures that the agent remains a memorable and engaging character over time.
Experimental Results
The authors benchmarked two Large Language Models (LLMs)—Chinese-Alpaca-Pro-7B and ChatGLM2-6B—across three settings: base model, few-shot prompting, and a combined "Fine-tuning + Prompting" approach.
Human evaluation on a 1–5 scale showed that the "Fine-tuning + Prompting" method significantly outperformed other settings. For the ChatGLM2-6B model, the Attraction score rose from 3.80 to 4.15, and the Stylization score (the ability to adhere to a requested style) improved from 3.45 to 3.80.
Most importantly, in "blind" tests where volunteers chatted with agents until they became bored, models trained on ContinuousChat sustained conversations for an average of 8.4 rounds, compared to only 6 rounds for the base models. This demonstrates a significant increase in user "stickiness" and engagement.
Conclusion
ContinuousChat represents a significant step toward more attractive and sustainable conversational agents. By enriching personas with depth, emotion, and stylistic variety, the dataset provides a roadmap for moving past the "Two Strangers" trap. While challenges remain in perfecting style-specific generation and cultural adaptation, the results show that fine-tuning on detailed, stylized data is a powerful tool for building agents that people actually want to keep talking to.
Summary updated