Abstract
Generating dialogue that consistently reflects a complex, multi-faceted persona remains a significant challenge in Natural Language Processing. Traditional methods often struggle with "personality sparsity" or the instability of natural language prompts. This paper introduces MIRACLE (MultIple PeRsonal Attributes Control within Latent-Space Energy-based Models), a framework that enables fine-grained, simultaneous control over multiple personality attributes. By leveraging a Conditional Variational Autoencoder (CVAE) with Latent-Space Energy-Based Models (EBMs) and ODE-based sampling, MIRACLE outperforms both specialized embedding methods and large language models like ChatGPT in maintaining nuanced personas while remaining computationally efficient.
Introduction
Creating chatbots that exhibit a consistent and specific personality is a long-standing goal. Current approaches typically fall into three categories: text-description-based (shallow descriptions), user-embedding-based (limited by sparse user data), and prompt-based LLMs (often unstable and imprecise for complex traits).
The core limitation of these methods is their inability to handle multi-faceted attribute control. Real human personality is a composition of various dimensions, such as language style, attitude, and internal characteristics. MIRACLE addresses this by disentangling these traits into a shared latent space, providing a "mixing board" for precise personality synthesis.
Methodology
The MIRACLE architecture is divided into three distinct stages designed to move from raw dialogue to controlled, personalized responses.
1. Disentangled Latent Space
The model utilizes a CVAE to map dialogues into a shared latent space. To ensure effective control, the authors introduce two novel loss functions:
- Aspect Classification Loss: This ensures that distinct traits (e.g., "pessimistic" vs. "optimistic") are clearly separated and identifiable.
- Attribute Distance Loss: This encourages the distributions of different attribute types (e.g., style and attitude) to overlap. This intersection is critical, as it allows the model to sample vectors that possess multiple traits simultaneously.
2. Latent Energy-Based Model (EBM)
Personality is modeled as a Product of Experts (PoE). Each desired attribute acts as an "expert" that assigns energy to points in the latent space. The system seeks a point with the lowest total energy, representing the optimal blend of all requested traits.
3. ODE Personalized Sampling
To navigate the energy landscape, the framework employs an Ordinary Differential Equation (ODE) sampling method. This approach "pulls" a latent vector toward areas of high attribute density before passing it to a GPT-2 based decoder to generate the final response.
Datasets and Evaluation
The researchers curated the MIRACLE Dataset, consisting of approximately 44,000 dialogue turns. They used ChatGPT to simulate conversations centered around seed topics from PersonaChat. While training focused on single attributes to ensure the model learned "pure" traits, evaluation was conducted on a separate test set of 4,600 instances featuring combinations of three attributes to test compositional ability.
Key Results
MIRACLE demonstrates significant improvements over existing baselines across several dimensions:
- Superior Personalization: MIRACLE achieved an average personalization accuracy of 92.75%, substantially higher than the strongest baseline, LMEDR (72.67%), and ChatGPT (80.46%) in multi-attribute scenarios.
- Coherence and Fluency: The model maintained high BLEU and NLI scores, ensuring that responses remained contextually relevant and fluent despite the heavy attribute constraints.
- Efficiency: MIRACLE is remarkably lightweight, training in just 3.2 hours on a single RTX 4090. Its inference speed (121s for 1000 items) is nearly ten times faster than calling LLM APIs like ChatGPT (>1000s).
Conclusion and Future Work
MIRACLE proves that mathematical control in a latent space is more precise and efficient for multi-attribute personalization than natural language prompting. While currently implemented with a DialoGPT-medium decoder, future iterations could utilize larger backbones like LLaMA to further enhance generation quality. Additionally, expanding the attribute scope to include the "Big Five" personality traits and integrating more robust safety constraints within the EBM framework are promising avenues for future research.
Summary updated