TL;DR
This paper introduces a novel "meta-lexicon" for French, designed for broad coverage and maintainability. Its architecture, based on property inheritance, allows for a formalism-independent description of lexical entries, enabling its use with diverse parsing frameworks like LFG and TAG.
What Problem Does This Paper Address?
Building comprehensive and accurate lexicons for natural language processing is a significant challenge. Lexicons need to cover a vast number of words (hundreds of thousands) and store complex morphological and syntactic information for each entry. This information must be compatible with various linguistic formalisms and parsing technologies. Traditional approaches often result in lexicons that are difficult to maintain, compact, and adaptable to different parsing frameworks. The authors aim to address this by proposing a more flexible and efficient lexicon architecture.
Methods
The core of the paper's contribution lies in the proposed "meta-lexicon" architecture. This architecture is divided into two phases:
- Morphological Phase: This phase involves creating a lexicon of inflected word forms. It starts with a list of lemmas and their inflection classes, along with conjugation tables. A conjugator then generates inflected forms. This is supplemented by manually entered inflected forms to handle variants and exceptions. Each inflected form is associated with a lemma, a morphological tag, and a morphosyntactic indicator that can influence parsing.
- Syntactic Phase: This phase focuses on the syntactic properties of lemmas. It uses a hierarchical structure of property inheritance. Lemmas are linked to syntactic "patterns," which are themselves defined by inheriting properties from other patterns or atomic syntactic properties. This allows for efficient modification and maintenance of syntactic information across related lexical entries. The authors also describe various automatic and semi-automatic techniques for acquiring, correcting, and completing the lexicon, including:
- Automatic Morphological Lexicon Acquisition: Leveraging corpus data and pre-defined inflectional classes to automatically identify and acquire inflected forms.
- Automatic Detection of Missing Words: Using spell checkers to identify unknown words in a corpus and suggest potential corrections or additions to the lexicon.
- Automatic Acquisition of Multi-word Expressions: Employing statistical techniques to identify common multi-word phrases.
- Automatic Detection of Syntactically Erroneous Entries: Analyzing parsing success rates to flag potentially problematic lexical entries.
- Targeted Automatic Acquisition of Syntactic Information: For specific cases like prepositions associated with verbs or support verbs followed by nouns.
Datasets
The paper mentions the use of a corpus from "Le Monde Diplomatique" for acquiring morphological forms. It also refers to the use of large corpora for detecting missing words and syntactically erroneous entries. The resulting lexicon is presented as the second version of "Lefff (Lexique des Formes Fléchies du Français)." No new benchmarks are introduced; rather, the focus is on the lexicon's creation and its application to existing parsing frameworks.
Key Insights and Ideas
- The "Meta-Lexicon" Concept: The key innovation is the idea of a "meta-lexicon" where lexical descriptions are independent of specific parsing formalisms. This is achieved through a property inheritance structure and a flexible representation of syntactic information.
- Property Inheritance for Efficiency: The hierarchical structure for syntactic information significantly improves maintainability. Changes to a property at a higher level automatically propagate to all related lexical entries, reducing redundancy and the effort required for corrections.
- Formalism Independence: By decoupling lexical descriptions from specific parsing frameworks (like LFG or TAG), the lexicon becomes reusable across different NLP systems, fostering interoperability and reducing development effort.
- Leveraging Automation for Scale: The paper highlights the necessity of automated and semi-automated techniques to manage the sheer scale of lexicon creation and maintenance, especially for broad-coverage lexicons.
Key Results
The paper describes a lexicon with over 400,000 inflected forms and over 600,000 entries. It demonstrates the successful application of this lexicon in building two distinct French parsers: one based on Tree-Adjoining Grammar (TAG) and another on Lexical Functional Grammar (LFG). This showcases the lexicon's versatility and its ability to support different linguistic formalisms.
Limitations and Future Work
The authors acknowledge that while automated techniques are employed, manual validation is still indispensable for ensuring the quality of the acquired lexical data, particularly for multi-word expressions. They also mention that some automated acquisition methods are more manual than others. Future work includes making the lexicon freely accessible. The paper also hints at potential improvements by suggesting that the current approach to defining syntactic patterns could be revisited to leverage derivational morphology.
Summary updated