Introduction
Open-domain dialogue systems frequently encounter the "1-to-n mapping" problem, where a single query (e.g., "How are you?") can elicit multiple valid responses ("I'm fine," "Not bad," "Pretty good"). Standard sequence-to-sequence (S2S) models often struggle with this diversity, defaulting to generic, "safe" responses like "I don't know." While Variational Autoencoders (VAEs) have been used to introduce diversity, they typically treat multiple references as independent samples, ignoring the underlying correlations between valid answers. This paper proposes a novel two-step architecture that explicitly models these correlations to improve both relevance and diversity.
The Two-Step Generation Framework
The core intuition behind the proposed approach is that it is mathematically more efficient for a model to first target the "mid-point" or commonality of all valid responses before branching out to specific variations.
Step One: Common Feature Extraction
The model first extracts a latent variable $c$ representing the shared features of a "response bag" (a set of valid responses for a given query). A mapping network transforms the query into this common feature space. To ensure $c$ captures meaningful semantic commonalities, a discriminator is employed in a minimax game, minimizing the discrepancy between generated intermediate representations and the average of ground-truth response embeddings.
Step Two: Response-Specific Generation
Building upon the common feature $c$, the model samples a second latent variable $z$ to capture the distinctive features of a specific response. This hierarchical approach constrains the search space, guiding the decoder toward the neighborhood of valid replies before refining the output into a fluent, specific sentence.
Technical Innovations
To prevent the common issue of "posterior collapse"—where the model ignores the latent variable—the authors introduce a Multi-reference Bag-of-Words (MBOW) loss. This loss forces the distinctive latent variable $z$ to capture the unique lexical characteristics of a specific response relative to others in the same bag. The framework is flexible and can be integrated with Conditional Variational Autoencoders (CVAE) or Wasserstein Autoencoders (WAE).
Experimental Results
The model was evaluated on the Weibo dataset, a large-scale Chinese conversation corpus. The test set was specifically curated to include four distinct, valid responses per query to test multi-reference performance.
Quantitative Performance
The proposed model ("Ours") consistently outperformed baselines (S2S, CVAE, WAE) across several key metrics. It showed significant gains in Distinctness (Dist-1/2), indicating the generation of more varied and less repetitive text, and Embedding Similarity, suggesting that the generated responses were more semantically aligned with human-authored references.
Human Evaluation
In blind human trials, the model was rated higher than S2S and CVAE baselines in both Relevance (3.22 vs. 3.10 for S2S) and Diversity (3.19 vs. 2.77 for S2S). Case studies revealed that the first step successfully captures general sentiments or topics, which the second step then refines into specific, contextually appropriate responses.
Conclusion
By treating multiple references as a correlated "bag" rather than isolated points, this two-step approach effectively addresses the 1-to-n mapping problem in dialogue generation. Modeling the "mid-point" of valid responses provides a stable foundation for generating diverse and relevant conversation, mitigating the "generic response syndrome" prevalent in simpler architectures.
Summary updated