Abstract
Identifying discourse structures and coherence relations in a piece of text is a fundamental task in natural language processing. The first step of this process is segmenting sentences into clause-like units called elementary discourse units (EDUs). Traditional solutions to discourse segmentation heavily rely on carefully designed features. In this demonstration, we present SegBot, a system to split a given piece of text into sequence of EDUs by using an end-to-end neural segmentation model. Our model does not require hand-crafted features or external knowledge except word embeddings, yet it outperforms state-of-the-art solutions to discourse segmentation.
Introduction
In Natural Language Processing, understanding the "flow" of a document requires discourse parsing. The critical first step in this process is discourse segmentation: breaking a sentence into Elementary Discourse Units (EDUs). For example, the sentence "[The bank also says] [it will use its network] [to channel investments]" contains three EDUs.
Historically, this has been treated as a "sequence labeling" problem where a model decides "yes" or "no" for a boundary at every single word. However, this approach suffers from extreme data sparsity—most words are not boundaries. Furthermore, traditional models relied heavily on complex, hand-crafted syntactic features. If the segmenter makes a mistake, those errors propagate through the entire discourse analysis pipeline, making downstream tasks like summarization or sentiment analysis less reliable. SEGBOT addresses these issues by treating segmentation as a sequence generation problem.
The SEGBOT Model
The authors propose SEGBOT, a model that shifts the paradigm from labeling to sequence generation.
The architecture consists of:
- Encoder: SEGBOT uses a bidirectional Gated Recurrent Unit (GRU) to process the input sentence, capturing context from both directions.
- Decoder: Instead of a standard classifier, the model uses a unidirectional GRU decoder equipped with a Pointer Network.
- The Pointer Mechanism: Unlike typical translation models that select from a fixed vocabulary, a pointer network "points" to specific positions in the input sequence. In SEGBOT, the decoder predicts the index of the next EDU boundary. Once a boundary is found, that position serves as the starting point for the next search.
This approach uses pre-trained GloVe word embeddings and requires no manual feature engineering or external linguistic knowledge.
Evaluation and Results
The model was evaluated using the RST Discourse Treebank (RST-DT), the gold standard corpus for Rhetorical Structure Theory, consisting of 385 Wall Street Journal articles.
SEGBOT achieved new state-of-the-art results, consistently outperforming both traditional feature-based models and more recent neural architectures. Key metrics include:
- F-score: SEGBOT reached 92.2%, a significant improvement over the previous neural baseline (BiLSTM-CRF) at 88.5%.
- Precision and Recall: The model achieved 91.6% precision and 92.8% recall, demonstrating a balanced ability to identify boundaries without over-segmenting.
- Comparative Performance: It outperformed established benchmarks such as the "DS" (Joty et al.) and "F&R" (Fisher and Roark) models.
Discussion and Future Work
The framing of segmentation as a sequence generation task allows the model to bypass the class imbalance problem inherent in sequence labeling. By predicting indices, the model naturally handles variable-sized "vocabularies" (boundary positions) at each decoding step.
While the model is generic, it was primarily tested on news text. Future research could explore its performance on informal text or technical documents. Additionally, integrating SEGBOT into a full end-to-end discourse parser that identifies relations (e.g., Contrast, Elaboration) between segments is a logical next step. The authors also suggest potential applications in larger-scale topic segmentation.
Summary updated