TL;DR
This paper introduces LOLA, a novel lexical database tool designed to streamline the creation and maintenance of lexicons for machine translation (MT) systems, specifically the LMT system. By providing a structured, multi-user environment with robust consistency checks and a focus on reusability, LOLA aims to overcome the significant bottleneck that lexical data represents in Natural Language Processing (NLP).
What Problem Does This Paper Address?
Building robust Natural Language Processing (NLP) systems, especially machine translation (MT) systems, hinges on the availability of comprehensive and well-structured lexical information. However, creating and maintaining these lexicons is a notoriously laborious and time-consuming task. Existing methods often involve manual hand-coding or the use of machine-readable dictionaries (MRDs) that require extensive revision and are not always suitable for NLP/MT purposes. This creates a significant bottleneck, hindering the development of new MT systems and prototypes. The authors highlight the need for a sophisticated tool that can guide and support lexicographers, making the process more efficient and the resulting data more reusable across different applications and language pairs.
Methods
The core of the paper is the description of the LOLA (Linguistic-Oriented Lexical database Approach) system. LOLA is built around a relational database (implemented using SQL/DS) that stores lexical data. Key components and methodologies include:
- Entity-Relationship (ER) Model: The conceptual design of the database is based on an ER model, which focuses on the structure of linguistic objects and their interrelations, aiming for data independence from specific application lexicons. This model allows for flexibility in representing lexical information and facilitates reusability.
- COLOLA (COder's interface to LOLA): This is a user-friendly interface designed for lexicographers. It provides a sequential menu-driven approach to hand-coding and modifying lexical data, mimicking the logical flow of lexicographic work. It supports operations like adding/deleting entries and features, and managing translation equivalents.
- Data Independence and Reusability: A central design principle is to make the lexical data as independent as possible from the specific format of application lexicons. This is achieved through the ER model and by storing detailed lexical information, allowing different MT/NLP systems to extract what they need.
- Consistency and Integrity Checks: LOLA incorporates automatic checks to ensure the validity of data and prevent inconsistencies, leveraging both the database management system and custom extensions.
- Multi-user Access: The system is designed to support simultaneous work by multiple lexicographers, with mechanisms to lock entries during updates to prevent conflicts.
- Conversion Programs: Several programs facilitate data flow:
- DB_TO_LMT: Extracts data from LOLA-DB and converts it into the LMT system's format.
- LMT_TO_DB: Loads existing LMT lexicons (in ELF format) into LOLA-DB.
- LDB_TO_DB: Converts data from machine-readable dictionaries (LDBs) into LOLA-DB, enabling the integration of external lexical resources.
Datasets
The paper does not explicitly mention specific datasets used for training or evaluation in the traditional sense. Instead, it focuses on the lexical data itself as the primary resource. The system is designed to handle:
- LMT lexicons: Existing lexicons for the LMT system, particularly in the External Lexical Format (ELF).
- Machine-Readable Dictionaries (MRDs): The system is designed to ingest data from MRDs, such as Collins German-English, English-German, Longman's Dictionary of Contemporary English, and Webster 7th Collegiate dictionary.
The paper emphasizes the creation and management of lexical data rather than empirical evaluation on a specific task.
Key Insights and Ideas
- Lexical Bottleneck is Real: The paper strongly reiterates that lexical data is a major hurdle in NLP/MT development, and a dedicated tool is crucial.
- Data Independence is Key to Reusability: By divorcing the database structure from application-specific formats, LOLA enables lexical data to be reused across different MT systems and even other NLP applications.
- Mimicking Lexicographic Workflow: The COLOLA interface is designed to align with how lexicographers naturally work, making the tool more intuitive and efficient.
- Leveraging Relational Databases: The choice of a relational DBMS (SQL/DS) provides essential features like multi-user access, consistency checking, and data integrity, which are vital for managing large lexicons.
- Bidirectional Translation Support: The ER model is designed to facilitate many-to-many relations between source and target language senses and slots, supporting bidirectional translation (e.g., German-English and English-German) without redundant descriptions.
Key Results
The paper focuses on the design and architecture of LOLA, rather than presenting quantitative results from empirical evaluations. The key "results" are the successful development of a system that:
- Provides a structured environment for lexicon construction and maintenance.
- Supports multi-user access for collaborative work.
- Offers robust consistency and integrity checks.
- Facilitates the conversion of data between LMT format, database format, and MRDs.
- Is designed with reusability across different language pairs and NLP applications as a core feature.
The paper highlights that the database scheme required only slight modifications to support new language pairs (English-Danish, English-Spanish), demonstrating the success of the data independence design.
Limitations and Future Work
- ER Model Limitations: The authors acknowledge that the ER model, while a compromise, may not possess the expressive power to cover all aspects of lexical description, especially complex constraints.
- Integration of External Lexicons: A significant challenge for future work is identifying and matching basic lexical categories (entries, homonyms, senses) across diverse lexical resources, which often have differing criteria and unclear definitions.
- Generalization of Conversion Programs: The conversion programs (DB_TO_LMT, LMT_TO_DB) need further generalization to abstract away from prototype-specific features of LMT for broader applicability.
- Terminology Workbench: A long-term goal is to extend LOLA into a terminology workbench, integrating MT and terminology management to leverage lexical knowledge from terminologists and translators for translation systems.
- Multilingual Database: The ultimate aim is a multilingual database where lexical knowledge for each language is represented only once, with application lexicons generated as needed.
Summary updated