Supervised Rhyme Detection with Siamese Recurrent Networks

Thomas Haider, Jonas KuhnCOLING 2018

TL;DR

This paper introduces a novel supervised approach for detecting rhymes using Siamese Recurrent Networks (SRNs), achieving near-perfect accuracy across German, English, and French. This method offers a scalable solution for analyzing rhyme patterns in large text corpora, opening new avenues for computational literary analysis and historical linguistics.

What Problem Does This Paper Address?

Rhyme is a fundamental element in poetry and song lyrics, yet its computational detection has been limited by small, handcrafted datasets and narrow definitions of "perfect" rhyme. Previous research often relied on unsupervised methods or struggled with imperfect rhymes, assonance, and variations in historical pronunciation. The authors aim to bridge this gap by developing a robust, supervised system capable of identifying rhyme with high accuracy across different languages and genres, enabling large-scale analysis of sonic patterns in literature.

Methods

The core of this paper is the application of Siamese Recurrent Networks (SRNs) for rhyme detection. SRNs are deep learning models that learn a similarity metric between two input sequences. In this context, each SRN consists of two identical recurrent sub-networks (specifically, bidirectional Long Short-Term Memory or biLSTM networks) that process individual words as sequences of characters. These networks generate vector representations, which are then compared using a cosine similarity measure. The model is trained on pairs of words, learning to distinguish between rhyming (positive) and non-rhyming (negative) pairs. For binary classification, a threshold of 0.5 on the cosine similarity is used. The authors also implemented an Expectation-Maximization (EM) baseline for comparison.

Datasets

The authors introduced three new corpora for rhyme detection:

  • DTR (German Text Archive Rhyme Gold): A diachronically balanced sample of 1,948 New High German poems, containing 8,147 stanzas and 13,785 positive rhyme pairs. This corpus was meticulously annotated for rhyme schema.
  • ANTI-K (Lyrik.Antikoerperchen): A collection of 156 poems from a website featuring student essays on poems, yielding 1,440 rhyme pairs.
  • HIPHOP-R (Hip-Hop Rhyme): 116 German Hip-Hop song texts, annotated for rhyme and assonance, contributing 2,489 rhyme pairs.

Additionally, they utilized existing datasets for English and French rhyme gold standards, which provided a substantial number of rhyme pairs for cross-lingual evaluation.

Key Insights and Ideas

  • SRNs Excel at Rhyme Detection: Siamese Recurrent Networks prove highly effective for learning nuanced similarity metrics between character sequences, making them well-suited for the complex task of rhyme detection, even for imperfect rhymes.
  • Language and Domain Independence: A single SRN model trained on a mixed-language and mixed-domain dataset (poetry and Hip-Hop) can achieve high accuracy across different languages and genres, demonstrating the generalizability of the approach.
  • Robustness to Noise: The SRN models show remarkable resilience to noise in the training data, maintaining high performance even with a significant percentage of incorrectly annotated rhyme pairs.
  • Domain Differences Matter: While SRNs perform well across domains, a model trained primarily on historical poetry struggles slightly with Hip-Hop lyrics, highlighting linguistic and stylistic differences that impact rhyme detection. Assonances are also less effectively captured by models focused on end-rhyme.

Key Results

  • Near-Perfect Accuracy: The SRN models achieve near-perfect accuracy (up to 97%) on rhyme detection tasks across German, English, and French datasets.
  • High Performance on German Corpora: For German, models trained on DTR and ANTI-K achieved accuracies of .96 and .94 respectively in binary classification.
  • Cross-Lingual Success: A language-independent model trained on data from three languages achieved .96 F1 score on German poetry and .92 F1 on Hip-Hop lyrics.
  • EM Baseline Performance: The EM baseline achieved accuracies around .71 for German stanzas, significantly lower than the SRN approach.

Limitations and Future Work

  • Assonance and Internal Rhyme: The current models, particularly when trained on end-rhyme focused datasets, do not effectively capture assonances or internal rhymes, which are prevalent in genres like Hip-Hop.
  • Domain Specificity: While language independence is achieved, performance can slightly degrade when a model trained on one domain (e.g., poetry) is applied to another (e.g., Hip-Hop) without specific fine-tuning.
  • Character Mapping Issues: Qualitative analysis revealed that SRNs can still make errors on specific character mappings or historical spelling variations.
  • Acceptable Noise Level: Future research should further investigate the maximum acceptable noise level in training data for SRNs.
  • Broader Applications: The authors suggest that this approach could be extended to analyze other sonic devices in literature and to reconstruct historical pronunciations.

Summary updated

BibTeX
@inproceedings{haider2018supervised,
  author = {Thomas Haider and Jonas Kuhn},
  title = {{Supervised Rhyme Detection with Siamese Recurrent Networks}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure