Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test

Qixin Wang, Tianyi Luo, Dong WangLecture Notes in Computer Science 2016 · DOI · OpenAlex

Abstract

Recent progress in neural learning demonstrated that machines can do well in regularized tasks, e.g., the game of Go. However, artistic activities such as poem generation are still widely regarded as human's special capability. In this paper, we demonstrate that a simple neural model can imitate human in some tasks of art generation. We particularly focus on traditional Chinese poetry, and show that machines can do as well as many contemporary poets and weakly pass the Feigenbaum Test, a variant of Turing test in professional domains. Our method is based on an attention-based recurrent neural network, which accepts a set of keywords as the theme and generates poems by looking at each keyword during the generation. A number of techniques are proposed to improve the model, including character vector initialization, attention to input and hybrid-style training. Compared to existing poetry generation methods, our model can generate much more theme-consistent and semantic-rich poems.

Introduction

Traditional Chinese poetry, specifically the quatrain (jueju), is a highly constrained literary form. It requires adhering to rigid structural rules—four lines of exactly five or seven characters—while maintaining specific rhyming schemes and tonal patterns (Ping/Ze). Beyond these "fetters," a poem must possess a "soul," characterized by thematic unity and emotional resonance.

Previous AI attempts at poetry generation often suffered from "theme drift," where the narrative coherence established in the first line would dissipate by the end of the quatrain. This paper addresses whether a neural model can master both the technical constraints and the creative requirement of thematic consistency.

Methodology: Attention-Based Generation

The authors propose a sequence-to-sequence framework based on Gated Recurrent Units (GRUs), enhanced with several novel mechanisms to ensure thematic integrity:

  • Keyword-to-Poem Generation: The model accepts a set of keywords as the "theme" rather than just a starting line, allowing for more controlled content.
  • Multiple-Attention Mechanism: The decoder utilizes an attention mechanism that monitors both the hidden states and the raw character vectors of the input keywords throughout the entire generation process.
  • Input Reconstruction: To anchor the model to its theme, the training process includes a step where the model must "reconstruct" the original keywords from its internal state after generating the poem, preventing the theme from being forgotten.
  • Hybrid-style Training: A single model is trained for both 5-character and 7-character poems using a style indicator vector, allowing it to leverage shared linguistic patterns across different formats.
The attention-based RNN architecture for keyword-to-poem generation.

To combat data sparsity, character embeddings were pre-trained on a massive corpus of nearly 300,000 ancient poems, providing the model with a rich semantic foundation before fine-tuning on a specific quatrain dataset.

Evaluation and the Feigenbaum Test

The model's performance was validated through both quantitative metrics and expert human evaluation. Quantitatively, the full model achieved BLEU scores of 0.330 (5-char) and 0.630 (7-char), significantly outperforming baseline RNN and SMT models.

To assess the "human-like" quality of the output, the authors employed the Feigenbaum Test, a domain-specific variant of the Turing Test. In a blind study involving 26 poetry experts, machine-generated poems were mixed with human-authored ones.

Results of the Feigenbaum Test: (a) Identification of human poems, (b) Identification of machine poems.

Key findings from the expert evaluation include:

  • The Fooling Rate: 31.02% of machine-generated poems were identified as human-written.
  • Expert Skepticism: The experts were highly critical, misidentifying 41.27% of actual human poems as machine-generated.
  • Quality Parity: Machine-generated 7-character poems received an average quality score of 3.17/5.0, very close to the 3.27/5.0 scored by human poets. Notably, machine poems took the top two spots in a ranked pool of the best 10 poems.

Conclusion

The results suggest that machines can effectively "dance in fetters," mastering the formal rules of classical Chinese poetry and even rivaling human experts in thematic consistency. While the ability to evoke deep aesthetic feeling ("Aesthesis") remains a primary human advantage, the model's success in the Feigenbaum Test marks a significant milestone in computational creativity. Future work will explore more complex genres like Song Iambics and the integration of Variational Autoencoders for enhanced natural innovation.

Summary updated

BibTeX
@inproceedings{wang2016can,
  author = {Qixin Wang and Tianyi Luo and Dong Wang},
  title = {{Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2016},
  doi = {10.1007/978-3-319-49685-6\_4},
}

Loading related content...

Figure