Producing Standard German Subtitles for Swiss German TV Content

Johanna Gerlach, Jonathan Mutal, Pierrette BouillonACL 2022

TL;DR

This paper tackles the challenge of automatically generating Standard German subtitles from Swiss German TV content. They compare Neural Machine Translation (NMT) and edit-based approaches, finding NMT to be more effective, especially when augmented with synthetically generated training data for specific linguistic phenomena.

What Problem Does This Paper Address?

Switzerland has a rich linguistic landscape, with two-thirds of the population speaking Swiss German, a spoken language with numerous dialects and no standardized written form. While Standard German is used for written communication, Swiss German is prevalent in TV content like news and talk shows. To make this content accessible to those who don't understand spoken Swiss German (e.g., due to hearing impairments or language preference), automatic subtitling is crucial. However, this process is complex, involving challenges like ASR errors, sentence boundary detection, handling spontaneous speech features, and, critically, transforming the syntactic differences between Swiss German and Standard German. This paper focuses on the latter challenge: bridging the syntactic gap to produce accurate Standard German subtitles.

Methods

The authors explore two main approaches for transforming normalized Swiss German ASR output into Standard German:

  • Neural Machine Translation (NMT): They employ a Transformer architecture with copy attention, a standard choice for tasks requiring minor text modifications. The system is trained on a large, low-quality corpus and then specialized on a smaller, high-quality corpus.
  • Edit-Based Systems: These systems predict types of edits (insertions, deletions, replacements) rather than directly generating words. They are trained on parallel data.

A key aspect of their methodology is the use of synthetic data. Recognizing the scarcity of parallel data for specific syntactic divergences, they generate synthetic parallel corpora. This involves using rule-based transformations on existing Standard German text (DE_PE and DE) to mimic Swiss German patterns, effectively creating training examples for phenomena like adjective phrase modifications, tun-periphrases, and particle usage. They also compare systems trained with and without this synthetic data (denoted by 's' prefix, e.g., sNMT, sEd).

Datasets

The study utilizes several datasets:

  • GSW_NORM: Normalized human transcriptions of Swiss German speech, representing an ideal ASR output.
  • DE (original subtitles): Original Standard German subtitles for TV shows.
  • DE_PE: Manually post-edited Standard German subtitles.
  • GSW_NORM-DE_PE: A corpus created by manually post-editing GSW_NORM into Standard German.
  • GSW_NORM-DE: An automatically aligned corpus of GSW_NORM and DE.
  • Synthetic Data (SDE_PE, SDE): Generated parallel data specifically targeting syntactic divergences, derived from DE_PE and DE.
  • Unseen ASR Output: A corpus of 5,000 segments of unseen real ASR output used for creating test suites.
  • Test Suites: Manually validated sets of examples (up to 50 per phenomenon) for evaluating specific syntactic divergences.

Key Insights and Ideas

  • NMT Outperforms Edit-Based: Neural Machine Translation systems generally perform better than edit-based systems in transforming Swiss German syntactic phenomena into Standard German.
  • Synthetic Data is Valuable: Generating synthetic parallel data using relatively simple rule-based transformations can be a cost-effective and valuable approach, especially when training data is scarce and transformations are straightforward.
  • Phenomenon Difficulty Varies: The success of transformations varies significantly depending on the specific syntactic phenomenon. Phenomena requiring simple reordering (like adverb-article order) are easier to correct than those requiring word-level transformations (like the tun-periphrase).
  • Synthetic Data Improves NMT: Augmenting NMT training with synthetic data leads to a higher number of transformed phenomena and a slight improvement in precision.
  • Edit-Based Systems Benefit Less from Synthetic Data: While synthetic data doesn't degrade the precision of edit-based systems, it doesn't yield as significant an improvement as it does for NMT.

Key Results

  • Syntactic Divergence Transformation: Human evaluations show that NMT systems achieve significantly higher correct transformation rates for most syntactic phenomena compared to edit-based systems. For example, for verbsAuxPP, NMT achieved 100% correct transformations with synthetic data (sNMT), while the edit-based system (sEd) achieved 63%.
  • Precision: NMT systems generally exhibit higher precision (correct edits relative to total edits) than edit-based systems. The addition of synthetic data slightly increases the precision for NMT.
  • Edit Counts: NMT systems perform more edits overall, but with higher precision, suggesting they are more effective at making necessary changes.

Limitations and Future Work

  • Limited Scope of Phenomena: The study focused on a subset of syntactic phenomena distinguishing Swiss German from Standard German.
  • Human Evaluation Constraints: The human evaluation was limited in scope due to practical constraints.
  • Orality Markers Ignored: The synthetic data generation did not account for orality markers (features of spontaneous speech) present in ASR output, which can complicate transformations.
  • Focus on Edits: The evaluation primarily focused on system performance in terms of performed edits, not necessarily on overall understandability or user satisfaction.

Future work includes extending the study to cover more phenomena, exploring different specialization settings, and conducting evaluations with target user groups to assess the impact on understandability and satisfaction.

Summary updated

BibTeX
@inproceedings{gerlach2022producing,
  author = {Johanna Gerlach and Jonathan Mutal and Pierrette Bouillon},
  title = {{Producing Standard German Subtitles for Swiss German TV Content}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure