Introduction
Short-text classification is a fundamental task in natural language processing, often applied to sentiment analysis or question answering. However, traditional approaches frequently treat utterances as isolated units, ignoring the rich contextual information inherent in human conversation. In dialog act recognition, for instance, the meaning of a simple "Yeah" depends entirely on whether it follows a question, an assertion, or a request. This paper introduces a neural framework that explicitly models the sequence of utterances to improve classification accuracy.
Utterance Representation
The first stage of the proposed model involves converting a single short-text utterance into a fixed-length vector representation ($s$). The authors evaluated two primary architectures for this "Short-text to Vector" (S2V) task: Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), specifically LSTMs.
For the RNN-based models, various pooling strategies—such as selecting the final hidden state or calculating the mean or maximum across all hidden states—were employed to summarize the sequence into a single vector. Interestingly, CNNs were found to slightly outperform LSTMs in representing short texts for this specific task.
Sequential Classification Framework
The core innovation of this work is the transition from isolated classification to sequential classification. Instead of predicting the class of an utterance based solely on its own representation, the model incorporates a "history" of preceding context. This is controlled by two hyperparameters:
- $d_1$: The number of previous utterance representations (vectors) included.
- $d_2$: The number of previous predicted class labels included.
These inputs are processed through a two-layer feedforward network. The authors discovered that incorporating text history ($d_1$) was generally more effective than label history ($d_2$). High-dimensional text vectors preserve more nuanced information, whereas labels are "lossy" and can lead to error propagation if a previous utterance was misclassified.
Experimental Results
The model was benchmarked against three major dialog act datasets: DSTC 4 (human-human dialogs), MRDA (multi-party meetings), and SwDA (telephone conversations). In all cases, the sequential models significantly outperformed non-sequential baselines.
- MRDA: The CNN-based sequential model achieved 84.6% accuracy, surpassing the previous graphical model baseline of 81.3%.
- SwDA: Reached 73.1% accuracy, a notable improvement over standard Hidden Markov Models (71.0%).
- DSTC 4: The LSTM-based model achieved 66.2% accuracy, representing a substantial leap from the SVM baseline of 57.0%.
Conclusion
By bridging the gap between deep learning and sequential modeling, this framework demonstrates that context is vital for short-text classification in dialog. While the model achieved state-of-the-art results, future work may explore more dynamic ways to model conversation flow beyond fixed history windows to further mitigate error propagation and handle longer-range dependencies.
Summary updated