Long text outline generation: Chinese text outline based on unsupervised framework and large language mode

Yan Yan, Yuanchi MaarXiv 2024 · DOI · arXiv

Abstract

Researchers have developed a hybrid, unsupervised framework that significantly improves how we generate outlines for ultra-long Chinese texts, such as epic novels. By combining Graph Attention Networks (GAT) and Markov chain principles to identify plot boundaries before using a Large Language Model (LLM) for summarization, the model achieves a staggering 87.5% segmentation accuracy—dramatically outperforming standalone models like GPT-4.

The Challenge of Ultra-Long Text Summarization

While Large Language Models (LLMs) are excellent at summarizing short articles, they often "lose the plot" when faced with million-word documents. For ultra-long fictional works, LLMs struggle to maintain a coherent thematic structure and frequently suffer from hallucinations or missed key events.

Existing outline generation methods typically focus on short-to-medium news snippets. The authors identify a critical gap: the lack of tools that can accurately segment the intricate, non-linear narrative structures of massive novels into meaningful plot points. Without precise segmentation, an LLM cannot produce a reliable, high-level roadmap of the story.

A Graph-Based Unsupervised Framework

The proposed pipeline moves away from pure text processing toward a structural, graph-based understanding of narrative. Instead of treating a chapter as a simple string of words, the system treats it as a graph where nodes represent entities (characters, locations) and edges represent syntactic dependency relationships.

The multi-step pipeline for long text outline generation, detailing the transition from raw text to graph-based segmentation and LLM summarization.

The framework follows a six-step process:

  1. Chapter-Level Pre-processing: Text is divided into chapters and processed via the Language Technology Platform (LTP) for Chinese Word Segmentation and Named Entity Recognition (NER).
  2. Entity Relationship Acquisition: Syntactic structure parsing identifies how characters and items interact.
  3. Graph Construction: Chapter data is converted into entity-relationship charts.
  4. Representation Learning: A Graph Autoencoder utilizing Graph Attention Layers (GAT) learns deep embeddings of these chapter graphs, capturing the "essence" of the narrative.
  5. Plot Boundary Division (The DMc Operator): Using Markov chains and path dependence, the model calculates the "distance" between chapter embeddings to mark plot boundaries.
  6. Outline Generation: The identified segments are fed into an LLM (like GPT-4) to generate concise themes.

Narrative Segmentation and Outline Generation

A core innovation of this research is the "Path Dependence" principle. By using Markov chains, the model acknowledges that a story's progression isn't just about the current chapter, but how it relates to the sequence of chapters preceding it. This prevents "jittery" segmentation and ensures the outline follows the natural flow of the story.

Conceptual overview of chapter division and outline generation using "The Count of Monte Cristo" as a representative example.

By handling the structural segmentation with a specialized Graph Neural Network (GNN), the LLM is freed from the burden of long-context navigation, which significantly reduces hallucinations and improves the logical consistency of the generated outline.

Key Results and Insights

The study introduces a new benchmark, the Chinese Long-Text Dataset, containing 31 ultra-long texts across genres like adventure, fantasy, and classics, with an average of 580 chapters per book.

Experimental results show a massive performance leap over state-of-the-art LLMs acting alone:

  • Segmentation Accuracy (ACC): The model achieved 87.5% accuracy, while GPT-4 only managed 26.1% and Llama-7b reached 37.3%.
  • Readability: Generated outlines were rated significantly higher for logical flow and coherence.
  • Consistency: The model maintained high performance even as text length increased, whereas standalone LLMs saw a sharp decline in accuracy as documents grew longer.

Conclusion

This research suggests that for ultra-long texts, the structural relationship between entities is a more stable indicator of a plot shift than raw semantic content. While the current multi-stage pipeline is highly effective, future work aims to integrate the unsupervised segmentation steps more tightly into LLM architectures and expand the framework to other languages.

Summary updated

BibTeX
@misc{yan2024long,
  author = {Yan Yan and Yuanchi Ma},
  title = {{Long text outline generation: Chinese text outline based on unsupervised framework and large language mode}},
  year = {2024},
  eprint = {2412.00810},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.00810},
}
Figure