The Rumour Mill: Making the Spread of Misinformation Explicit and Tangible

Nanna Inie, Jeanette Falk Olesen, Leon Derczynski

Abstract

Misinformation spread presents a technological and social threat to society. With the advance of AI-based language models, automatically generated texts have become difficult to identify and easy to create at scale. We present "The Rumour Mill", a playful art piece, designed as a commentary on the spread of rumours and automatically-generated misinformation. The mill is a tabletop interactive machine, which invites a user to experience the process of creating believable text by interacting with different tangible controls on the mill. The user manipulates visible parameters to adjust the genre and type of an automatically generated text rumour. The Rumour Mill is a physical demonstration of the state of current technology and its ability to generate and manipulate natural language text, and of the act of starting and spreading rumours.

Introduction

As AI-based language models become more sophisticated, they have reached a point where automatically generated text is often indistinguishable from human-written content. This creates a massive scalability problem for misinformation: bad actors can generate believable, high-volume fake news at a very low cost, while the effort required for human fact-checkers to verify and debunk these claims remains high. The authors describe this as a "Sisyphean" task due to the asymmetry between the ease of generation and the difficulty of verification.

This process is usually "covert"—happening behind closed doors or via anonymous bot farms. There is a lack of public understanding regarding how these models work and what they are capable of. "The Rumour Mill" addresses this by bringing the generation process into the physical world, making it overt, visible, and even "awkward" to operate, thereby demystifying the technology.

System Design and Interaction

The Rumour Mill is a tabletop interactive machine housed in a wooden and plexiglass box. The system uses a two-step natural language generation (NLG) pipeline to produce content. The user influences the output based on three physical inputs that map directly to model parameters.

The control panel of The Rumour Mill, featuring dials for 'Wackiness' and 'Genre'.
  • Wackiness: A potentiometer maps to the model's "temperature" (stochastic sampling) setting. Low wackiness produces safe, high-probability word choices; high wackiness leads to unusual, unpredictable, and "creative" phrasing.
  • Genre: A 12-step switch allows users to select the "flavor" of the rumour, such as Conspiracy Theory, CNN Business, Celebrity Gossip, or Russia Today.
  • When: A toggle switch sets the time period (Past, Present, or Future) by injecting specific date-related control codes into the model.

Data and Modeling

The generation pipeline consists of two primary models:

  1. Headline Generation: The system uses a GPT-2 instance fine-tuned on a dataset of headlines from fact-checking sites like Snopes, Politifact, and Emergent. This ensures the output mimics the specific "hook" style of viral misinformation.
  2. Body Text Generation: The generated headline is fed as a seed into CTRL (Conditional Transformer Language Model). CTRL is designed for "controllable" generation, leveraging a massive 140GB corpus. The authors utilize CTRL's built-in "control codes" to steer the generation toward the user-selected genres.

Discussion: Tangibility and Transparency

By making the machine large and the process of "milling" a rumour a physical act (turning a rotary lever), the authors transform a digital threat into a social performance. It is difficult to spread a rumour covertly when physically cranking a noisy machine in a public space.

One of the most elegant translations in the project is mapping the technical concept of "stochastic sampling temperature" to a physical dial labeled "wackiness." This provides a non-technical audience with an intuitive grasp of how AI randomness works. Furthermore, to prevent the machine from becoming an actual source of harm, every printout is clearly labeled as a generated rumour, following "critical design" principles—creating an object that provokes thought without causing the very problem it critiques.

Results and Conclusion

The project resulted in a functional, portable installation that successfully generates fluent, genre-consistent misinformation. The system can produce text that matches specific styles, such as a "Celebrity Gossip" rumour about a wedding or a "Politics" rumour about the State Department.

Sample outputs from The Rumour Mill showing generated rumours with associated metadata.

The Mill provides a playful yet critical way for users to experience the state of current NLP technology. It moves the conversation from abstract fear of "AI bots" to a concrete understanding of how parameters like genre and randomness influence the "believability" of a lie. While the Mill helps people identify current AI-generated text, the authors acknowledge that as models improve, identifying them will only become more difficult, suggesting a need for ongoing iterations of such educational tools.

Summary updated

Abstract

Misinformation spread presents a technological and social threat to society. With the advance of AI-based language models, automatically generated texts have become difficult to identify and easy to create at scale. We present "The Rumour Mill", a playful art piece, designed as a commentary on the spread of rumours and automatically-generated misinformation. The mill is a tabletop interactive machine, which invites a user to experience the process of creating believable text by interacting with different tangible controls on the mill. The user manipulates visible parameters to adjust the genre and type of an automatically generated text rumour. The Rumour Mill is a physical demonstration of the state of current technology and its ability to generate and manipulate natural language text, and of the act of starting and spreading rumours.
BibTeX
@inproceedings{inie2020rumour,
  author = {Nanna Inie and Jeanette Falk Olesen and Leon Derczynski},
  title = {{The Rumour Mill: Making the Spread of Misinformation Explicit and Tangible}},
  booktitle = {CHI Extended Abstracts},
  year = {2020},
  doi = {10.1145/3334480.3383159},
}
Figure