World WordNet Database Structure: An Efficient Schema for Storing Information of WordNets of the World

Hanumant Harichandra Redkar, Sudha Baban Bhingardive, Diptesh Kanojia, Pushpak Bhattacharyya

Abstract

This paper introduces the World WordNet Database Structure (WWDS), a standardized relational database schema designed to unify the storage and linking of WordNet data across diverse languages. By partitioning data into a central, language-independent "master" database and multiple language-specific "spoke" databases, WWDS facilitates efficient cross-lingual querying and eliminates the fragmentation caused by disparate file formats. This architecture supports advanced Natural Language Processing (NLP) tasks like machine translation and multilingual information retrieval by providing a universal pivot for conceptual mapping.

Introduction

WordNet is a foundational lexical database in Natural Language Processing (NLP) that groups words into cognitive synonyms called "synsets." Since the creation of the original Princeton English WordNet, organizations worldwide have developed WordNets for dozens of other languages, such as GermaNet, IndoWordNet, and EuroWordNet.

However, these individual WordNets use highly fragmented storage formats. Some utilize flat text files, others use XML, and many rely on custom relational databases. Flat files and XML are notoriously inefficient for complex querying, data manipulation, and cross-referencing. This lack of standardization makes it difficult for researchers to build multi-lingual NLP systems, as they must write custom parsers and mapping logic for every new language they integrate. The World WordNet Database Structure (WWDS) is proposed as a solution to this fragmentation.

The WWDS Architecture

To solve the problem of data fragmentation, the authors introduce a standardized relational database schema heavily inspired by the existing IndoWordNet structure. The architecture relies on a clear separation of concerns through a hub-and-spoke model.

The hub-and-spoke architecture of WWDS, connecting a central master database to multiple language-specific WordNets.

As shown in the architecture diagram, the system is divided into two primary components:

  • wordnet_master (The Hub): A single, central database that stores language-independent information. This includes universal semantic relations and overarching ontology details that remain conceptually true regardless of the spoken language.
  • wordnet_<language> (The Spokes): Multiple language-specific databases (e.g., wordnet_hindi, wordnet_german, wordnet_english). These store language-dependent data, such as specific words, localized synsets, and language-specific lexical relations.

Schema Design and Implementation

The core of the WWDS is its detailed relational schema, which allows for complex joins and efficient data retrieval. To tie the disparate databases together, the schema utilizes an inter-lingual synset ID (il_synset_id) located in a master mapping table (wn_master_inter_lingual_synset). This ID acts as a universal pivot key, linking equivalent concepts across all individual language databases.

The detailed Entity-Relationship (ER) diagram of the WWDS, showing the separation between language-specific tables and the master database tables.

The Entity-Relationship diagram illustrates how the wn_synset table in a language-specific database connects to the wordnet_master tables. This structure allows an NLP application to identify the ID for a concept (e.g., "dog") in the master database and instantly pull corresponding synsets in French, Hindi, or Japanese using standard SQL joins, without needing direct language-to-language mapping files.

Key Insights and Advantages

The transition from flat files to a normalized relational structure offers several key benefits:

  • Efficiency and Reduced Redundancy: By splitting language-independent data from language-dependent data, the schema reduces data redundancy and simplifies multi-lingual updates.
  • Universal Translation via Pivot IDs: The use of a single il_synset_id acts as a universal translator at the database level, streamlining cross-lingual information retrieval.
  • Optimized Querying: Relational databases allow researchers to leverage highly optimized queries, which is crucial for large-scale data mining and complex NLP applications.

Conclusion and Future Work

The WWDS provides a robust framework for centralized access to cross-lingual synset data. While the current schema is designed to be universal, the authors acknowledge that individual languages often have unique linguistic quirks that may require further refinement of the model.

Future work will focus on studying specific features of individual languages to revise the WWDS schema and expanding the structure to store supplementary linguistic information, such as text corpora, annotated data, and morphological features alongside core WordNet data.

Summary updated

Abstract

WordNet is an online lexical resource which expresses unique concepts in a language. English WordNet is the first WordNet which was developed at Princeton University. Over a period of time, many language WordNets were developed by various organizations all over the world. It has always been a challenge to store the WordNet data. Some WordNets are stored using file system and some WordNets are stored using different database models. In this paper, we present the World WordNet Database Structure which can be used to efficiently store the WordNet information of all languages of the World. This design can be adapted by most language WordNets to store information such as synset data, semantic and lexical relations, ontology details, language specific features, linguistic information, etc. An attempt is made to develop Application Programming Interfaces to manipulate the data from these databases. This database structure can help in various Natural Language Processing applications like Multilingual Information Retrieval, Word Sense Disambiguation, Machine Translation, etc.
BibTeX
@inproceedings{redkar2015world,
  author = {Hanumant Harichandra Redkar and Sudha Baban Bhingardive and Diptesh Kanojia and Pushpak Bhattacharyya},
  title = {{World WordNet Database Structure: An Efficient Schema for Storing Information of WordNets of the World}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2015},
  doi = {10.1609/aaai.v29i1.9276},
}
Figure