TL;DR
This paper presents a novel finite-state transducer for Spanish that efficiently handles both inflectional and derivational morphology, processing tens of millions of word forms from a lexicon of around 55,000 base words. The system's elegance lies in its simple data structures and interpretation, enabling fast analysis and generation with a surprisingly small storage overhead.
What Problem Does This Paper Address?
Computational morphology aims to model the relationship between a word's base form (lemma) and its various inflected and derived forms. For languages like Spanish, with rich morphology, this involves handling numerous variations in spelling and meaning based on grammatical features. Traditional approaches often involve complex automata composition or parallel processing. The authors sought a simpler, more unified approach to represent these word relations, specifically for Spanish, that would be both efficient and easy to implement. They wanted to build a system that could accurately generate and analyze a vast number of Spanish word forms from a relatively small lexicon.
Methods
The core of the system is an n-ary finite transducer, conceptualized as a labeled directed graph. Each arc in this graph is labeled with an n-tuple of strings, representing the relationship between different linguistic levels (e.g., surface spelling, lemma, morphosyntactic features). For Spanish, they primarily use 3-tuples, often reduced to 2-tuples by concatenating the lemma and morphosyntactic features.
A key innovation is the arc-list compiler. Instead of manually defining the graph, this compiler generates it from a lexical table that mirrors dictionary entries. This compiler applies morphological rules to derive stem allomorphs and endings for verbs, nouns, and adjectives. The system handles:
- Stem Allomorphy: Spanish verbs exhibit significant stem variations (e.g., "ten-", "teng-", "tien-", "tend-", "tuv-" for "tener"). The compiler generates these allomorphs based on conjugation class and morphophonological rules.
- Orthographic Rules: Spanish has specific orthographic changes (e.g., "c" to "qu" before "e" or "i"). These are incorporated into the arc generation.
- Affixes and Clitics: Sublexicons for endings and clitics are integrated.
- Special Cases: Reflexive verbs and defective verbs are handled with specific treatments.
The system's interpretation is a simple traversal of this arc list, making it efficient.
Datasets
- Collins Spanish Dictionary: This served as the primary source for lexical items and morphological information. The system was built to process approximately 55,000 base words from this dictionary.
- EFE Newswire Text: The system was evaluated on its ability to process text from the EFE newswire, covering about 95% of Spanish text from this source.
Key Insights and Ideas
- Simplicity of N-ary Transducers: The paper demonstrates that n-ary finite transducers, when applied to languages like Spanish, offer a remarkably simple and unified way to represent complex morphological relationships. This contrasts with more intricate automata composition methods.
- Efficient Storage Overhead: The system achieves a low storage overhead (less than 5%) for representing predictable stem allomorphy and affixes, making it practical for large lexicons.
- Symmetric Analysis and Generation: The finite-state nature of the system allows it to function symmetrically for both analyzing (recognizing) and generating word forms, using the same underlying description.
- Compiler-Driven Approach: Automating the creation of the morphological graph via a compiler significantly reduces the manual effort required to build such a system.
Key Results
- The processor can handle tens of millions of inflected and derived Spanish word forms from a base lexicon of about 55,000 words.
- The system processes several hundred words per second on a Sun4 workstation, indicating high efficiency.
- The storage overhead for compiling predictable morphological variations is less than 5% of the total lexicon size.
- The system covers approximately 95% of Spanish text from the EFE newswire.
Limitations and Future Work
- Language Specificity: While powerful for Spanish, the authors acknowledge that languages with more complex compounding (like German) might present special challenges.
- Future Work: The authors plan to extend this model to other Romance languages and to explore its application to German, particularly its compound word structures. They also mention its integration into a Spanish parser and a Spanish newswire browser.
Summary updated