LMF Reloaded

Laurent Romary, Mohamed Khemakhem, Fahad Shahbaz Khan, Jack Bowers, Nicoletta Calzolari, Monte George, Mandy Pet, Piotr Bański

Abstract

Lexical Markup Framework (LMF) or ISO 24613 is a de jure standard that provides a framework for modelling and encoding lexical information in retrodigitised print dictionaries and NLP lexical databases. An in-depth review is currently underway within the standardisation subcommittee, ISO-TC37/SC4/WG4, to find a more modular, flexible and durable follow-up to the original LMF standard published in 2008. In this paper, we present major improvements implemented in the new version of LMF, focusing on modularity and the formal modeling of etymological data.

Introduction: Addressing the Monolithic Legacy

The original Lexical Markup Framework (LMF), published in 2008, was designed to standardize dictionary modeling for both human use and Natural Language Processing (NLP). However, it suffered from significant limitations: it was a rigid, "one-size-fits-all" model that became overly complex, it lacked coverage for diachronic (etymological) information, and its serialization was not easily compatible with the Text Encoding Initiative (TEI), the de facto standard for digital humanities.

To address these issues, the ISO standardization sub-committee has developed a "reloaded" version of the standard designed to be more flexible, durable, and modular.

A Modular Architecture for Lexical Data

The fundamental shift in the new LMF is its transition from a single document to a multi-part system. The standard is now split into seven distinct modules, including a Core Model (ISO 24613-1), a Machine Readable Dictionary (MRD) model, and specific modules for Etymology, Serializations, and linguistic extensions (Syntax, Semantics, Morphology).

This decoupling allows developers to implement only the necessary components for their specific use case. Furthermore, the introduction of a simplified CrossREF mechanism replaces several obsolete, complex classes, making the meta-model leaner and easier to implement for complex entries like multi-word expressions (MWEs).

Figure 1: Core and MRD model representation for the entry "center", demonstrating modularity and the CrossREF mechanism linking to "dead center".

The Etymology Breakthrough

One of the most significant additions to the framework is the Diachrony-Etymology module (ISO 24613-3). By introducing classes such as Etymon, Cognate, and EtyLink, the standard finally allows researchers to model the "life story" of a word in a machine-readable way.

Using the entry for "center" from Ernest Klein’s A Comprehensive Etymological Dictionary of the English Language as a test case, the model successfully tracks the word's journey from its reconstructed Proto-Indo-European roots through Ancient Greek, Latin, and Old French into modern English.

Figure 2: The new Etymology model tracking the complex linguistic history and borrowing chain of the word "center".

Conclusion and Future Directions

The reloaded LMF successfully demonstrates that a modular approach can handle both complex modern lexical entries and deep historical data while moving toward full TEI compatibility via the TEI Lex-0 subset. While work continues on the Syntax, Semantics, and Morphology modules, the current framework provides a robust foundation for bridging the gap between traditional lexicography and modern NLP engineering. The ultimate success of the standard will depend on its adoption by the broader community over ad-hoc, custom formats.

Summary updated

Abstract

Lexical Markup Framework (LMF) or ISO 24613 [1] is a de jure standard that provides a framework for modelling and encoding lexical information in retrodigitised print dictionaries and NLP lexical databases. An in-depth review is currently underway within the standardisation subcommittee , ISO-TC37/SC4/WG4, to find a more modular, flexible and durable follow up to the original LMF standard published in 2008. In this paper we will present some of the major improvements which have so far been implemented in the new version of LMF.
BibTeX
@misc{romary2019lmf,
  author = {Laurent Romary and Mohamed Khemakhem and Fahad Shahbaz Khan and Jack Bowers and Nicoletta Calzolari and Monte George and Mandy Pet and Piotr Bański},
  title = {{LMF Reloaded}},
  year = {2019},
  eprint = {1906.02136},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1906.02136},
}

Loading related content...

Figure