We Are What We Repeatedly Do: Inducing and Deploying Habitual Schemas in Persona-Based Responses

Benjamin Kane, Lenhart K. SchubertEMNLP 2023 · DOI

Introduction

Most persona-based dialogue systems treat personality as a static collection of isolated facts, such as "I have a dog" or "I enjoy hiking." While Large Language Models (LLMs) can utilize these facts, the resulting conversations often feel shallow and lack the depth of human identity, which is fundamentally revealed through habits and narratives. Researchers from the University of Rochester address this gap by introducing habitual schemas—structured understandings of the goals, sub-events, and sensory details that define a persona's recurring activities.

The Habitual Schema Framework

The authors propose a multi-stage pipeline designed to transform simple persona facts into rich, lived-in conversational behavior. This process moves beyond mere fact retrieval to active narrative construction.

Overview of the habitual schema induction and deployment pipeline.
  1. Schema Induction (Bootstrapping): Starting with a basic persona fact (e.g., "I bake pies"), the system prompts an LLM to generate a "generic passage" describing how that activity typically unfolds.
  2. Structuring: The LLM distills this narrative into a formal event schema containing preconditions (e.g., "I have ingredients"), static conditions (e.g., "The oven is hot"), goals, and step-by-step episodes.
  3. Retrieval: During a live dialogue, the system employs a Sentence Transformer to identify and retrieve the most relevant schema based on the user's input.
  4. Generation: The retrieved schema is provided as in-context knowledge. The researchers evaluated two generation modes: Unconstrained (UNCS), where the LLM generates freely, and Few-shot Paraphrase (PARA), which constrains the LLM to weave specific schema details into a target response.

Experimental Setup

The framework was evaluated using the PersonaChat dataset, a standard benchmark containing over 131,000 utterances. The researchers compared their schema-augmented models (UNCS and PARA) against a standard LLM baseline (BASE) and human-generated "gold" responses.

Key Results

The inclusion of habitual schemas led to significant improvements across both quantitative metrics and human qualitative assessments.

Quantitative comparison of Diversity and Controllability metrics across different models.
  • Diversity: The schema-based methods significantly outperformed the baseline in output richness. The entropy (ENTR) score, measuring vocabulary diversity, rose from 2.91 in the baseline to 3.84 in the PARA model.
  • Controllability: The PARA method achieved substantially higher similarity to human "gold" responses, with BLEU scores jumping from 1.25 to 8.60 and ROUGE-L scores nearly doubling.
  • Human Preference: In head-to-head evaluations, human judges found the schema-augmented responses to be significantly more engaging and relevant than the baseline.

Discussion and Future Work

A key insight from this research is that "banal" details—such as putting on walking shoes or preheating an oven—are essential for making AI agents feel physically present and grounded in a routine. By moving from facts to scripts, the system allows for a more consistent and narrative-driven persona.

However, challenges remain. The authors noted occasional hallucinations in pronoun logic and "affective mismatches," where long narrative responses might feel inappropriate following a user's emotional disclosure. Future work will focus on capturing "atypical" events—unique memories that distinguish an individual from a generic category—and reducing the computational overhead associated with complex schema retrieval.

Summary updated

BibTeX
@inproceedings{kane2023we,
  author = {Benjamin Kane and Lenhart K. Schubert},
  title = {{We Are What We Repeatedly Do: Inducing and Deploying Habitual Schemas in Persona-Based Responses}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.678},
}
Figure