Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations

Hao Yang, Hongyuan Lu, Xinhua Zeng, Yang Liu, Xiang Zhang, Yumeng Zhang, Shan Huang, Yiran Wei, Wai Lam

Abstract

In the rapidly evolving field of natural language processing, dialogue systems primarily employ a single-step dialogue paradigm. Although this paradigm is commonly adopted, it lacks the depth and fluidity of human interactions and does not appear natural. We introduce a novel Step-by-Step Dialogue Paradigm (Stephanie), designed to mimic the ongoing dynamic nature of human conversations. By employing a dual learning strategy and a further-split post-editing method, we generated and utilized a high-quality step-by-step dialogue dataset to fine-tune existing large language models, enabling them to perform step-by-step dialogues. We thoroughly present Stephanie. Tailored automatic and human evaluations are conducted to assess its effectiveness compared to the traditional single-step dialogue paradigm. We will release code, Stephanie datasets, and Stephanie LLMs to facilitate the future of chatbot eras.

Introduction

Most modern dialogue systems rely on a Single-Step Dialogue Paradigm. When a user interacts with a bot, the system processes the input and returns a single, often lengthy, block of text. While efficient for task-oriented queries, this approach fails to capture the essence of social conversation. In real-life digital communication, such as WhatsApp or iMessage, humans rarely send one giant paragraph; instead, we send a series of shorter, interconnected thoughts in sequence.

Comparison between the traditional single-step dialogue system and the proposed Stephanie paradigm.

The authors argue that the current single-step approach lacks the depth, fluidity, and "human-likeness" required for AI companions or social chatbots. This paper introduces Stephanie, a paradigm where the AI sends multiple, shorter, coherent messages in sequence, mimicking the "bursty" nature of human messaging to significantly improve engagement and naturalness.

Methodology

The researchers developed a multi-stage pipeline to transform standard Large Language Models (LLMs) into effective "step-by-step" conversationalists:

  1. Dual Learning Strategy: To teach the model the difference between "stiff" and "natural" flows, the team utilized contrastive prompting. The model was provided with both positive learning objectives (high-quality step-by-step dialogues) and negative learning objectives (traditional single-step dialogues) based on the same themes and personas.
  2. Further-Split Post-Editing: Initial model outputs were often still too dense. The authors introduced a refinement step where dialogues were further broken down according to the natural emotional progression of a conversation, ensuring the "bursts" of messages felt logical rather than random.
  3. Specific Fine-Tuning: Using the newly generated data, they fine-tuned Llama3-8b. Special delimiters were introduced in the training data to teach the model exactly when to "break" its response into multiple messages.
The Stephanie input-output pipeline, highlighting the dual learning strategy and contrastive prompting.

Datasets

The authors built upon the well-known PERSONA-CHAT dataset, which contains dialogues grounded in specific character traits.

  • Original Source: 8,939 dialogue segments from PERSONA-CHAT.
  • Stephanie Dataset: A newly created, high-quality dataset consisting of 5,457 segments. Unlike traditional datasets that follow an A-B-A-B turn-taking structure, the Stephanie dataset follows an A-(B1, B2, B3)-A structure, where one "turn" consists of multiple messages from a single speaker.

Key Insights and Results

The "Burstiness" of Humanity

The core insight of this research is that naturalness in digital communication is defined not just by what is said, but by how it is paced. By increasing the Average Consecutive Message Count (ACMC), the AI feels more like a friend and less like a search engine. Traditional models have an ACMC of 1.0 (one message per turn), whereas Stephanie achieved an ACMC of 2.51, much closer to human-like messaging patterns.

Performance Metrics

The authors compared Stephanie against several baselines, including GPT-4 and Llama3-70b, using both automated metrics and human judges:

  • Human Engagement: On a 1-5 scale, Stephanie earned an Engaging score of 4.06, significantly higher than the original single-step model's 3.13.
  • Naturalness: In human evaluations, Stephanie's "Natural" score (3.97) beat out the single-step baseline (2.97) by a wide margin.
  • Lexical Diversity: Stephanie showed higher Distinct-N scores, particularly in longer n-grams, suggesting the step-by-step format encourages more creative and less repetitive language.
  • Conciseness: Stephanie reduced the average words per message to 5.87, compared to 11.77-13.67 in baseline models, favoring the short-form communication typical of instant messaging.

Conclusion

Stephanie represents a significant step toward more emotionally resonant and human-like AI companions. By prioritizing conversational pacing and sequential messaging, the paradigm enhances user engagement and perceived naturalness. While the authors acknowledge limitations regarding the scale of human evaluation and potential safety risks inherent in LLMs, they believe this paradigm could define the future of social chatbots and gaming NPCs. The researchers plan to release the Stephanie datasets and fine-tuned models to the community to facilitate further research in this area.

Summary updated

Abstract

In the rapidly evolving field of natural language processing, dialogue systems primarily employ a single-step dialogue paradigm. Although this paradigm is commonly adopted, it lacks the depth and fluidity of human interactions and does not appear natural. We introduce a novel **Step**-by-Step Dialogue Paradigm (Stephanie), designed to mimic the ongoing dynamic nature of human conversations. By employing a dual learning strategy and a further-split post-editing method, we generated and utilized a high-quality step-by-step dialogue dataset to fine-tune existing large language models, enabling them to perform step-by-step dialogues. We thoroughly present Stephanie. Tailored automatic and human evaluations are conducted to assess its effectiveness compared to the traditional single-step dialogue paradigm. We will release code, Stephanie datasets, and Stephanie LLMs to facilitate the future of chatbot eras.
BibTeX
@inproceedings{yang2025stephanie,
  author = {Hao Yang and Hongyuan Lu and Xinhua Zeng and Yang Liu and Xiang Zhang and Haoran Yang and Yumeng Zhang and Shan Huang and Yiran Wei and Wai Lam},
  title = {{Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=mDPwh3zTyl},
}

Loading related content...

Figure