MoliAIre, a theatrical agent which speaks like Molière's characters

Guillaume Grosjean, Tristan Cazenave, Baptiste Roziere, Anna PappaBase 2022

TL;DR

MoliAIre is a specialized conversational agent fine-tuned to mimic the theatrical style of the French playwright Molière. By combining multi-author pre-training with a clever "reverse generation" technique to handle rhymes, the researchers created a model capable of improvising 17th-century dialogue that respects both historical linguistic style and poetic constraints.

What Problem Does This Paper Address?

While Large Language Models (LLMs) like GPT-3 have become incredibly proficient at general conversation, they often struggle with specific artistic "voices" or strict formal constraints like those found in classical theater. Most conversational agents are trained on modern web data, making them ill-suited for niche literary applications.

The authors identify a "real scarcity" of AI applications in the field of theater. Specifically, they tackle the challenge of generating text that adheres to the unique vocabulary and grammar of 17th-century French, while also attempting to solve the structural difficulty of generating rhymed verse (like alexandrines) in an autoregressive framework.

Methods

The researchers built MoliAIre using a phased approach, starting with a baseline and moving toward more complex architectural tweaks:

  1. Baseline Model: They started with GPT-fr, a 1-billion parameter Transformer model pre-trained on a large French corpus. They fine-tuned this on a dataset of Molière’s plays, structured as "USER vs. MoliAIre" dialogue exchanges.
  2. Multi-Author Training: To improve the model's robustness, they adopted a strategy from the translation field: training on multiple similar sources. They added 502 plays from other 17th-century dramatists to the training set. This "second pre-training" gave the model a broader understanding of the era's linguistic style before it was finally "honed" on Molière’s specific works.
  3. Reverse Language Modeling (MoliAIre-VERSE): Standard LLMs generate text from left to right. This makes rhyming difficult because the model has to "land" on a specific rhyming word at the end of a line after the start of the line has already been determined. To fix this, the authors trained a version that generates lines in reverse. By starting with the last word of the line (the rhyme), the model can then "backfill" the rest of the sentence to satisfy the rhyme constraint more easily.
  4. Inference: The model uses top-k sampling (k=40) to maintain a balance between coherence and the "improvisational effect" necessary for theater.

Datasets

  • Primary Corpus: 32 plays by Molière, totaling 15,283 dialogue cues.
  • Extended Corpus: 502 additional 17th-century plays, bringing the total to 189,905 lines.
  • Pre-training Data: The underlying GPT-fr model was originally trained on a 1-billion token French web corpus.

Key Insights and Ideas

  • The "Era" as a Feature: One of the most interesting findings is that training on Molière’s contemporaries (the multi-author model) significantly improved performance on Molière-specific tasks. This suggests that for niche styles, "data augmentation" via chronologically similar authors is more effective than simply using more modern data.
  • Solving Rhyme via Inversion: The "aha moment" for poetic generation was realizing that autoregressive constraints are the enemy of rhyming. By reversing the token sequence, the rhyme becomes the input or the first choice, rather than a difficult-to-hit target at the end of a sequence.
  • Style Persistence: The model demonstrated a surprising ability to stay "in character." Even when prompted with modern French, MoliAIre responded in its 17th-century style rather than switching to modern vernacular.

Key Results

  • Perplexity: The multi-author approach successfully lowered the model's perplexity (a measure of how well the model predicts the sample) from 14.88 (baseline) to 11.52.
  • BLEU Scores: Using Molière’s original lines as a reference, the multi-author model achieved a mean BLEU score of 35.56, outperforming the mono-author baseline (30.63).
  • Qualitative Success: The MoliAIre-VERSE model showed a high facility for generating alexandrines and rhymes. However, it occasionally "cheated" by rhyming a word with itself.
  • Improvisation: The authors noted that the model rarely "overfits"—it doesn't just parrot Molière’s existing lines but generates new, stylistically consistent responses.

Limitations and Future Work

Despite its stylistic flair, MoliAIre faces several hurdles:

  • Coherence: Like many smaller LLMs, it can occasionally contradict itself or lose the logical thread of a long conversation.
  • Topic Sensitivity: If a user brings up modern topics far removed from the 17th-century experience, the model’s performance degrades.
  • Evaluation Metrics: The authors acknowledge that BLEU scores are a blunt instrument for creative work. They suggest that future work should involve human-based evaluation by experts in theater and literature.
  • Hybrid Modes: Currently, the model struggles to switch seamlessly between prose and verse; future iterations could focus on a model that handles both formats with equal grace.

Reference: Grosjean, G., Cazenave, T., Roziere, B., & Pappa, A. (2022). MoliAIre, a theatrical agent which speaks like Molière’s characters. Proceedings of the XXV Generative Art Conference, GA2022.

Summary updated

Abstract

This project introduces an artistic dimension to automatic conversational agents through the implementation of MoliAIre, a t heatrical agent based on a generative language model which gives replicas like Molière’s characters. A first version, used as a b aseline, shows the artistic potential of the model. We describe the results by exploring two directions that significantly improve the performances of the model. On the first one we train the model on several authors having a similar style to Molière to improve the performances. On the second one, a r everse generation method favours the generation of rhymes. Every model presented in this paper is available online for testing: https://www.lamsade.dauphine.fr/molierelebot.
BibTeX
@inproceedings{grosjean2022moliaire,
  author = {Guillaume Grosjean and Tristan Cazenave and Baptiste Roziere and Anna Pappa},
  title = {{MoliAIre, a theatrical agent which speaks like Molière's characters}},
  booktitle = {Base},
  year = {2022},
}
Figure