TL;DR
This paper introduces SCOPE, a novel approach for generating similes from literal sentences by leveraging common sense knowledge and a pre-trained language model (BART). SCOPE not only creates more evocative machine-generated stories but also demonstrates impressive performance compared to human experts and existing baseline systems.
What Problem Does This Paper Address?
Figurative language, particularly similes, plays a crucial role in human imagination and communication, adding depth and vividness to descriptions. However, automatically generating creative and meaningful similes is a challenging task. Existing research has primarily focused on simile detection rather than generation. The core problem addressed here is the lack of a robust method for generating novel, contextually relevant, and evocative similes from simple literal statements, which is essential for applications like creative writing assistance and enhancing the expressiveness of machine-generated text.
Methods
The authors propose a two-step approach called SCOPE (Style transfer through COmmonsense PropErty):
- Automatic Parallel Corpus Creation: They construct a large parallel corpus of literal-simile pairs by scraping Reddit. Similes are first identified using the phrase "like a." To create the "literal" counterpart, they leverage structured common sense knowledge from COMET (Bosselut et al., 2019) to transform the simile's vehicle into its implied property. For example, "Love is like a unicorn" is transformed into "Love is rare." This process is further refined using a pre-trained language model (GPT) for ranking literal versions and a grammatical error correction model.
- Seq2Seq Model for Simile Generation: They fine-tune a pre-trained BART (Lewis et al., 2019) sequence-to-sequence model on the constructed parallel corpus. BART's encoder takes the literal sentence as input, and its decoder is trained to generate the corresponding simile. At inference time, top-k sampling is used for generation.
Datasets
- Training Data: A custom parallel corpus of 87,843 self-labeled similes collected from Reddit (specifically the WRITINGPROMPTS and FUNNY subreddits). This corpus was split into 82,697 samples for training and 5,146 for validation.
- Test Data: 150 literal sentences were scraped from the WRITINGPROMPTS subreddit, with the last word restricted to adverbs or adjectives. These were used for evaluation.
Key Insights and Ideas
- Common Sense is Key for Simile Generation: The paper strongly emphasizes that generating effective similes requires more than just syntactic transformation; it necessitates understanding the common sense properties that link two concepts. Their method explicitly incorporates this by using COMET to infer properties.
- Style Transfer Framework is Suitable: Framing simile generation as a style transfer problem, where the goal is to transform a literal description into a more figurative one, proves effective.
- BART's Power for Creative Generation: Fine-tuning a powerful pre-trained model like BART on a custom dataset of literal-simile pairs allows it to learn the nuances of creative language generation.
- Novelty is Achievable: The approach successfully generates a high percentage of novel similes, indicating good generalization capabilities beyond the training data.
- Similes Enhance Machine-Generated Stories: Replacing literal sentences in machine-generated stories with similes produced by their model significantly improves evocativeness and reader acceptance.
Key Results
- Automatic Metrics: SCOPE achieved the best scores across BLEU-1, BLEU-2, and BERTScore metrics compared to baselines like BART, Retrieval (RTRVL), and Metaphor Masking (META M).
- Novelty: The model generates novel similes 88% of the time.
- Human Evaluation:
- SCOPE generated similes that were preferred over two literary experts 37% of the time and over three baseline systems (including META M) 71% of the time in pairwise comparisons.
- In a task-based evaluation, stories with SCOPE-generated similes were preferred by human judges 42% of the time compared to stories without similes.
- Storytelling Impact: Replacing literal sentences with similes in GPT-2 generated stories led to a 42% win rate in human evaluations for overall quality and evocativeness when compared to original GPT-2 stories and GPT-2 stories post-processed with META M.
Limitations and Future Work
- Noise in Datasets: The authors acknowledge that their automatically constructed parallel corpus may contain some noise due to distant supervision, though they found their method still performed well.
- Contextual Relevance: While their method incorporates common sense, ensuring perfect relevance to the specific context of a literal sentence remains an ongoing challenge.
- Future Directions: The authors suggest exploring other knowledge bases, applying their simile generation approach to different creative NLG tasks (like sarcasm and hyperbole), and further investigating the interplay between similes and metaphors.
Summary updated