Hierarchical Methods for a Unified Approach to Discourse, Domain, and Style in Neural Conversational Models

João SedocAAAI 2018 · DOI

Introduction

Modern neural conversational models often struggle with long-term coherence and stylistic consistency. Standard Sequence-to-Sequence (Seq2Seq) architectures treat dialogue as a flat mapping between input and output strings, failing to capture the multi-scale nature of human speech. This paper proposes a hierarchical framework that explicitly models three pillars of dialogue: domain (topic), discourse (dialog acts), and style (personality).

The Hierarchical Approach

The proposed system decomposes the conversational task into three specialized components to move beyond simple string mapping:

1. Factored Response Prediction (DOM-Seq2Seq)

To handle diverse topics, the model employs a domain classifier that routes user utterances to specific, domain-targeted Seq2Seq generators. A high-level re-ranker then selects the most contextually appropriate response from the generated candidates based on the current conversation history.

The DOM-Seq2Seq architecture, which factors response generation by domain and uses a re-ranker for final selection.

2. Coherent Conversational Tree

This component manages the "flow" of dialogue by predicting the next "dialog act" (e.g., acknowledgment vs. follow-up question). It utilizes a hybrid of multilayer Hidden Markov Models (HMMs) and multiresolution Seq2Seq models, allowing the system to condition word-level generation on high-level conversational goals.

3. Response Style Shifting

To maintain a consistent persona, the framework uses a dual-Seq2Seq approach combined with a modified Word Graph algorithm. This allows the model to "shift" the vocabulary of a general response toward a target style (e.g., a specific character's personality) without altering the underlying meaning.

Key Insights: The Power of Utterance-Level Modeling

A central thesis of this work is that modeling conversations at the utterance level is more effective than word-level modeling for maintaining long-term coherence. By capturing high-level statistics ($h^{(2)}$ in the architecture) for each turn, the model can guide lower-level word generation ($h^{(1)}$) more effectively.

Hierarchical architecture showing the relationship between high-level utterance representations ($h^{(2)}$) and low-level word generation ($h^{(1)}$).

This hybridization of modern deep learning with classical probabilistic structures, such as spectral HMMs and tensor factorization, provides a more interpretable and controllable framework for AI conversation. By splitting the model into domain-specific generators, the system avoids the "average" response problem, where a model trained on general data ends up providing generic, uninteresting answers.

Conclusion and Future Work

The research demonstrates that hierarchical modeling significantly improves topic consistency and conversational relevance. While the current framework successfully captures domain and style, the proactive generation of discourse cues (e.g., "however," "consequently") remains an area for future development. Integrating these cues will be critical for achieving truly natural, logical flow in automated dialogue systems over the coming months.

Summary updated

BibTeX
@inproceedings{sedoc2018hierarchical,
  author = {João Sedoc},
  title = {{Hierarchical Methods for a Unified Approach to Discourse, Domain, and Style in Neural Conversational Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11366},
}
Figure