Identification of Parallel Sentences in Comparable Monolingual Corpora from Different Registers

Rémi Cardon, Natalia GrabarEMNLP 2018

TL;DR

This paper tackles the challenge of automatically identifying and aligning parallel sentences in comparable monolingual corpora, specifically focusing on French texts from different registers (specialized vs. simplified). They developed a feature-rich approach that achieves high accuracy, paving the way for better text simplification tools and cross-lingual NLP applications.

What Problem Does This Paper Address?

Text simplification aims to make complex texts more accessible. A crucial step in this process is identifying pairs of sentences that convey the same information but differ in complexity – essentially, a "difficult" sentence and its "simplified" counterpart. While parallel corpora (sentence-by-sentence translations) are ideal for this, they are scarce, especially for languages other than English. This paper addresses this gap by proposing a method to find these "parallel" sentences within comparable monolingual corpora. Comparable corpora consist of texts on similar topics but not direct translations, making them easier to obtain. The challenge lies in the fact that simplified sentences might not share many exact words with their specialized counterparts, requiring more sophisticated alignment techniques than simple word matching.

Methods

The authors propose a multi-stage approach:

  1. Pre-processing: Documents are POS-tagged and lemmatized using TreeTagger, and then segmented into sentences.
  2. Feature Generation: A comprehensive set of features is engineered to capture various aspects of sentence similarity and difference. These include:
  3. Lexical Overlap: Counts of common non-stopwords and stopwords.
  4. Inclusion: Percentage of words from one sentence present in another.
  5. Length Differences: Simple and average length differences between sentences.
  6. N-grams: Common character-based bigrams and trigrams to capture sub-word patterns.
  7. Word-based Similarity: Cosine, Dice, and Jaccard similarity scores, both with uniform word weights and with TF-IDF weighting.
  8. Edit Distance: Character-based and word-based minimal edit distances (Levenshtein).
  9. Automatic Alignment: A set of linear classifiers (Perceptron, MLP, LDA, QDA, Logistic Regression, SGD, LinSVM) are trained on these features to classify sentence pairs as either "aligned" (parallel) or "non-aligned."

Datasets

The primary datasets used are three comparable monolingual corpora in French from the biomedical domain:

  • Drug Corpus: Drug information for professionals and patients.
  • Scientific Corpus: Summaries of high-evidence health-related articles and their simplified versions.
  • Encyclopedia Corpus: Wikipedia articles and their simplified counterparts from Vikidia.

For evaluation, they also utilize datasets from the SemEval Semantic Textual Similarity (STS) competitions (2012-2014) in English. They adapt the STS task (which scores similarity from 0-5) into a binary classification task (aligned/non-aligned) by setting similarity thresholds.

Key Insights and Ideas

  • Leveraging Comparable Corpora: The core idea is that even without direct translation, comparable corpora with different registers (specialized vs. simplified) can provide valuable signals for identifying parallel sentences.
  • Feature Richness is Key: The study demonstrates that a combination of diverse features, including lexical, length-based, n-gram, and edit distance measures, is crucial for robust alignment.
  • Logistic Regression Shines: Among the tested classifiers, Logistic Regression emerged as the best performer on their French data, achieving a strong balance between precision and recall.
  • Cross-lingual Transferability: The approach shows promising transferability to English, suggesting that the developed features and models can be applied across languages, which is vital for broader NLP applications.
  • Simplification Implies Similarity: The work implicitly highlights that text simplification, while altering wording, often maintains a high degree of semantic similarity, which their alignment method can detect.

Key Results

  • French Data: The proposed method achieved up to 0.93 F-measure on the French comparable corpora, with a good balance between Precision and Recall. Logistic Regression was identified as the best classifier.
  • English STS Data (Monolingual): When applied to English STS data, the system achieved up to 0.90 F-measure using a similarity threshold of 4.5.
  • English STS Data (Cross-lingual): Training on French data and testing on English STS data yielded up to 0.82 F-measure, demonstrating good cross-lingual transferability.

Limitations and Future Work

  • Limitations:
    • The authors acknowledge that the STS data and their French data differ in task definition (binary vs. graded similarity) and domain, which introduces limitations to direct comparison.
  • The TF-IDF weighting feature, surprisingly, did not perform well on their data, contrary to some previous work.
  • Future Work:
    • Exploit the best models to enrich the set of parallel sentences for French text simplification.
  • Apply the approach to other NLP applications.
  • Explore additional features, such as word embeddings, to better handle lexical variations.
  • Further evaluate the approach on data from other languages.

Summary updated

BibTeX
@inproceedings{cardon2018identification,
  author = {Rémi Cardon and Natalia Grabar},
  title = {{Identification of Parallel Sentences in Comparable Monolingual Corpora from Different Registers}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure