Analysis of Rhythmic Phrasing: Feature Engineering vs. Representation Learning for Classifying Readout Poetry

Timo Baumann, Hussein Hussein, Burkhard Meyer-SickendiekCOLING 2018

TL;DR

This paper explores classifying rhythmic patterns in German readout poetry using machine learning. They found that a neural network approach, learning representations directly from text, speech, and audio, significantly outperformed traditional methods relying on manually engineered features, achieving an impressive f-measure of 0.85. This suggests that deep learning can effectively capture subtle rhythmic nuances in poetry that are difficult to hand-engineer.

What Problem Does This Paper Address?

Traditional methods for analyzing poetry meter often fall short in capturing the full complexity of rhythmic structures in modern and postmodern poetry. These poems frequently deviate from strict metrical patterns, employing more fluid and nuanced rhythmic devices. Existing digital tools, like the "Metricalizer," are limited in their scope. This paper addresses this gap by investigating two specific rhythmic patterns, "Parlando" and "Variable Foot," prevalent in German readout poetry, and comparing two distinct machine learning approaches for their classification: one based on manually engineered features and another using representation learning with neural networks. The goal is to determine which approach is more effective at capturing the subtle rhythmic characteristics that differentiate these poetic styles.

Methods

The authors compare two primary machine learning approaches:

  • Manually Engineered Features: This method involves extracting specific features from the text and speech of the poems. Key features include:
  • Pause Features: Analyzing the length and distribution of pauses in the audio recordings, particularly at line endings, as the "Variable Foot" pattern is theorized to involve longer pauses.
  • Parser Features: Utilizing natural language processing (NLP) to analyze the syntactic structure of the text, such as identifying part-of-speech tags and sentence structure, to capture textual cues related to phrasing.
  • Combined Features: A set that merges both pause and parser features. They experimented with several classification algorithms and settled on AdaBoostM1.

  • Neural Network-based Representation Learning: This approach uses a hierarchical attention network. The network takes various inputs, including:

  • Text character sequences.
  • Acoustic features (MFCCs) from the speech.
  • Acoustic features from pauses following lines. The network learns its own representations from these inputs, aiming to automatically discover features relevant for classification without explicit manual engineering. This method uses a bidirectional recurrent network with gated recurrent units and inner attention.

Datasets

The study uses a corpus of German readout poems from the lyrikline (www.lyrikline.org) website. This dataset includes audio recordings by the poets themselves and their corresponding texts. For this specific study, the authors collected 68 poems from 24 poets, with 34 poems classified as "Parlando" and 34 as "Variable Foot." Due to the relatively small dataset size, they employed 10-fold cross-validation for their experiments.

Key Insights and Ideas

  • Neural Networks Outperform Manual Engineering: The most significant insight is that the neural network approach, which learns representations automatically, significantly outperforms the manually engineered feature approach. This highlights the power of deep learning in capturing complex, subtle patterns that are difficult for humans to explicitly define and extract.
  • Speech and Pause Information is Crucial: While text-based features are useful, incorporating speech and pause information into the neural network model led to a substantial improvement in classification performance. This underscores the importance of the auditory dimension in understanding poetic rhythm.
  • Theoretical Expectations vs. Empirical Results: Interestingly, the manually engineered pause features, which were theoretically expected to be highly indicative of the "Variable Foot" pattern, did not yield the best results. Instead, parser features performed better within the manual feature set, and the neural network ultimately surpassed both. This suggests that theoretical assumptions about rhythmic cues might not always translate directly into the most effective computational features.
  • Rhythm is More Than Meter: The paper reinforces the idea that modern poetry's rhythmic richness extends far beyond traditional metrical analysis, requiring more sophisticated computational approaches.

Key Results

The paper reports the following weighted f-measure results:

  • Manually Engineered Features:
    • Pause features: 0.69
    • Parser features: 0.69
    • Pause + Parser features: 0.62
  • Neural Network and Representation Learning:
    • Text only: 0.65
    • Text + Speech: 0.85
    • Text + Speech + Pause: 0.85

The neural network approach, particularly when incorporating speech and pause information, achieved a peak f-measure of 0.85, significantly outperforming the best manual feature engineering result (0.69).

Limitations and Future Work

Limitations:

  • Dataset Size: The authors acknowledge the relatively small dataset size (68 poems) and used 10-fold cross-validation to mitigate this.
  • Alignment Accuracy: While they used a text-speech aligner, the accuracy of these alignments was not explicitly checked.
  • Feature Engineering Nuances: The performance of manual features might be further limited by the specific features chosen and the classification algorithms used.

Future Work:

  • Understanding Learned Representations: The authors aim to better understand what specific poetic aspects are encoded within the representations learned by the neural network.
  • Expanding to More Patterns: The philological scholar has defined 18 rhythmical patterns, and the authors plan to analyze and classify these additional patterns.
  • Exploring Other Languages: The current study focuses on German poetry; future work could extend to other languages.

Summary updated

BibTeX
@inproceedings{baumann2018analysis,
  author = {Timo Baumann and Hussein Hussein and Burkhard Meyer-Sickendiek},
  title = {{Analysis of Rhythmic Phrasing: Feature Engineering vs. Representation Learning for Classifying Readout Poetry}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure