Challenges of language change and variation: towards an extended treebank of Medieval French

Mathilde Regnault, Sophie Prévost, Éric Villemonte de la Clergerie2019 · DOI

Abstract

In order to automatically extend a treebank of Old French (9th–13th c.) with new texts in Old and Middle French (14th–15th c.), we need to adapt tools for syntactic annotation. However, these stages of French are subjected to great variation, and parsing historical texts remains an issue. We chose to adapt a symbolic system, the French Metagrammar (FRMG), and develop a lexicon comparable to the Lefff lexicon for Old and Middle French. The final goal of our project is to model the evolution of language through the whole period of Medieval French (9th–15th c.).

Introduction

Historical linguistics faces a significant challenge in the digital age: the lack of standardized, large-scale corpora for non-modern languages. This project aims to bridge this gap by extending the Syntactic Reference Corpus of Medieval French (SRCMF). The goal is to expand the current treebank into a comprehensive one-million-word resource covering both Old French (9th–13th c.) and Middle French (14th–15th c.).

Parsing historical languages is notoriously difficult for standard Natural Language Processing (NLP) tools. Unlike modern French, Medieval French is characterized by high dialectal variation, inconsistent orthography, and "null subjects" (omitting the pronoun). Furthermore, word order was significantly more fluid; while modern French is strictly Subject-Verb-Object (SVO), Medieval French frequently employed VSO or SOV configurations.

Methodology: The FRMG Pipeline

Due to the scarcity of annotated data for Middle French, which limits the effectiveness of purely statistical parsers, the researchers adopted a symbolic (rule-based) approach. The core of the pipeline is the French Metagrammar (FRMG), a high-level system based on Tree Adjoining Grammar (TAG). TAG utilizes "elementary trees" as modular building blocks that combine to represent complex syntactic structures.

The symbolic parsing pipeline for Medieval French, integrating the FRMG metagrammar and specialized lexicons.

The pipeline integrates several specialized components:

  • OFrLex (Old French Lexicon): A specialized lexicon mapping varied medieval spellings to standardized entries, analogous to the modern Lefff lexicon used for contemporary French.
  • Hybrid Disambiguation: While the grammar generates multiple possible analyses for a sentence, a statistical model trained on existing treebanks is used to select the most probable parse.

Structural Innovation: Flattening the Tree

A primary innovation in this project is the move away from rigid hierarchical trees to better accommodate free word order. Traditional hierarchical structures often fail when sentence components shift positions. To address this, the authors introduced a "flatter" representation using a special ## node. This allows sibling nodes—such as the Subject (Arg0), Verb (VN), and Object (Arg1)—to appear in varying sequences without violating the underlying grammatical logic.

An example of a flattened tree structure for the sentence "ele ainme li," utilizing the ## node to accommodate flexible word order.

Insights and Future Directions

The project adopts a "continuum" approach, using a single broad metagrammar for the entire 9th–15th century period. Temporal or dialectal metadata can then be used to "restrain" the parser, effectively filtering out constructions inappropriate for a specific text's era. Additionally, the grammar's constraints have been intentionally "loosened" to permit historical inconsistencies, such as variable verb agreement with collective nouns (e.g., gent).

While this is a work in progress, the architectural success in flattening the FRMG grammar marks a significant step forward. Future work will involve a full-scale evaluation against statistical baselines—such as the Mate parser, which achieved a Labelled Attachment Score (LAS) of 76.04 on Old French—and the implementation of "error mining" to refine rules for rare dialectal quirks. Ultimately, this hybrid approach aims to provide a robust framework for modeling the long-term evolution of the French language.

Summary updated

Abstract

In order to automatically extend a treebank of Old French (9 th -13 th c.) with new texts in Old and Middle French (14 th -15 th c.), we need to adapt tools for syntactic annotation. However, these stages of French are subjected to great variation, and parsing historical texts remains an issue. We chose to adapt a symbolic system, the French Metagrammar (FRMG), and develop a lexicon comparable to the Lefff lexicon for Old and Middle French. The final goal of our project is to model the evolution of language through the whole period of Medieval French (9 th -15 th c.).
BibTeX
@misc{regnault2019challenges,
  author = {Mathilde Regnault and Sophie Prévost and Éric Villemonte de la Clergerie},
  title = {{Challenges of language change and variation: towards an extended treebank of Medieval French}},
  year = {2019},
  doi = {10.18653/v1/w19-7816},
}
Figure