TL;DR
This paper introduces a novel data-driven system for generating portmanteaux (compound words like "frenemy") using a unique multitape Finite State Transducer (FST). By training on existing portmanteaux, their system achieves a respectable 45% exact match rate, offering a more robust approach than previous rule-based methods.
What Problem Does This Paper Address?
Creating new words by blending existing ones, known as portmanteau generation, is a common and often creative linguistic phenomenon. While humans do this intuitively, building an automated system is challenging. Previous attempts have been limited by either strictly matching word segments (like "brunch" from "breakfast" + "lunch") or relying on extensive, hand-coded linguistic knowledge. This paper tackles the problem of generating portmanteaux by considering both the phonetic compatibility of component words and the blending process itself, aiming for a data-driven approach that can handle more complex phonetic fusions.
Methods
The core of the system is a novel multitape Finite State Transducer (FST). Unlike standard FSTs that handle one input and one output, this multitape FST is designed for a 2-input, 1-output scenario, specifically to take the pronunciations of two component words (W¹pron and W²pron) and produce the pronunciation of the portmanteau (PMpron).
The overall system is structured as an FSM cascade:
- FST A: Converts input words (W¹, W²) into their pronunciations (W¹pron, W²pron) using the CMU Pronouncing Dictionary.
- wFST B: This is the key multitape FST that performs the phonetic blending of W¹pron and W²pron into PMpron. It's trained using a statistical model.
- wFST C: Converts the generated PMpron back into a written word (PM') using grapheme-phoneme alignments.
- wFSA D, FSA E1,2: These act as "mini language models" to refine the generated PM' by favoring letter trigrams from the original words and rejecting inputs that don't align with the component words.
The system is trained using an Expectation-Maximization (EM) algorithm to learn weights for the FSTs, particularly for the multitape wFST B. They explore three different methods for learning phoneme alignment probabilities: conditional, joint, and mixed alignment. Finally, a reranking step combines outputs from multiple models and uses learned features to improve the final selection.
Datasets
The primary dataset consists of portmanteaux and their component words obtained from Wikipedia and Wiktionary. The authors filtered these to ensure they met specific constraints (e.g., exactly two component words). They also used manual annotations of portmanteau pronunciations for mid-cascade evaluation. The data was split for 10-fold cross-validation, with 8 folds for training, 1 for development, and 1 for testing.
Key Insights and Ideas
- Multitape FSTs are powerful for multi-input/output NLP tasks: The paper demonstrates the utility of a 2-input, 1-output FST for a problem that traditional FSTs struggle with, opening doors for similar architectures in other areas.
- Phonetic blending is crucial for natural portmanteaux: The success of the system hinges on its ability to blend pronunciations, not just written forms, allowing for more complex and natural-sounding word fusions.
- Data-driven approaches outperform rule-based systems: By training on real-world examples, the system learns patterns that are difficult to hand-code, leading to better performance.
- Reranking significantly boosts performance: While initial generation might have lower exact match rates, reranking based on various features dramatically improves the quality of the final output.
Key Results
- The system achieves 45% exact matches on portmanteau generation in cross-validated experiments.
- The average Levenshtein edit distance for the generated portmanteaux is 1.59.
- Over 90% of the 1000-best generated lists contain the correct pronunciation, indicating the system is good at generating plausible candidates even if the top one isn't perfect.
- A baseline system that merges W¹ and W² at the first shared phoneme achieved only 33% exact matches for PMpron and 25% for PM, highlighting the advantage of the FST approach.
- Qualitative examples show successful generation of words like "affluenza," "joggling," and "tofurkey," with some hypotheses even being considered superior to the "gold standard."
Limitations and Future Work
- Limitations: The paper acknowledges that the system still produces unacceptable outputs ("js," "mman") and that the exact match rate of 45% indicates room for improvement. The system is primarily focused on the blending aspect, not the selection of component words.
- Future Work: The authors are interested in developing systems that can select component words for portmanteaux and also reconstruct component words from portmanteaux. They also plan to explore other applications for multi-input/output models.
Summary updated