Abstract
Misinformation spread presents a technological and social threat to society. With the advance of AI-based language models, automatically generated texts have become difficult to identify and easy to create at scale. We present "The Rumour Mill", a playful art piece, designed as a commentary on the spread of rumours and automatically-generated misinformation. The mill is a tabletop interactive machine, which invites a user to experience the process of creating believable text by interacting with different tangible controls on the mill. The user manipulates visible parameters to adjust the genre and type of an automatically generated text rumour. The Rumour Mill is a physical demonstration of the state of current technology and its ability to generate and manipulate natural language text, and of the act of starting and spreading rumours.
Introduction
As AI-based language models become more sophisticated, they have reached a point where automatically generated text is often indistinguishable from human-written content. This creates a massive scalability problem for misinformation: bad actors can generate believable, high-volume fake news at a very low cost, while the effort required for human fact-checkers to verify and debunk these claims remains high. The authors describe this as a "Sisyphean" task due to the asymmetry between the ease of generation and the difficulty of verification.
This process is usually "covert"—happening behind closed doors or via anonymous bot farms. There is a lack of public understanding regarding how these models work and what they are capable of. "The Rumour Mill" addresses this by bringing the generation process into the physical world, making it overt, visible, and even "awkward" to operate, thereby demystifying the technology.
System Design and Interaction
The Rumour Mill is a tabletop interactive machine housed in a wooden and plexiglass box. The system uses a two-step natural language generation (NLG) pipeline to produce content. The user influences the output based on three physical inputs that map directly to model parameters.
- Wackiness: A potentiometer maps to the model's "temperature" (stochastic sampling) setting. Low wackiness produces safe, high-probability word choices; high wackiness leads to unusual, unpredictable, and "creative" phrasing.
- Genre: A 12-step switch allows users to select the "flavor" of the rumour, such as Conspiracy Theory, CNN Business, Celebrity Gossip, or Russia Today.
- When: A toggle switch sets the time period (Past, Present, or Future) by injecting specific date-related control codes into the model.
Data and Modeling
The generation pipeline consists of two primary models:
- Headline Generation: The system uses a GPT-2 instance fine-tuned on a dataset of headlines from fact-checking sites like Snopes, Politifact, and Emergent. This ensures the output mimics the specific "hook" style of viral misinformation.
- Body Text Generation: The generated headline is fed as a seed into CTRL (Conditional Transformer Language Model). CTRL is designed for "controllable" generation, leveraging a massive 140GB corpus. The authors utilize CTRL's built-in "control codes" to steer the generation toward the user-selected genres.
Discussion: Tangibility and Transparency
By making the machine large and the process of "milling" a rumour a physical act (turning a rotary lever), the authors transform a digital threat into a social performance. It is difficult to spread a rumour covertly when physically cranking a noisy machine in a public space.
One of the most elegant translations in the project is mapping the technical concept of "stochastic sampling temperature" to a physical dial labeled "wackiness." This provides a non-technical audience with an intuitive grasp of how AI randomness works. Furthermore, to prevent the machine from becoming an actual source of harm, every printout is clearly labeled as a generated rumour, following "critical design" principles—creating an object that provokes thought without causing the very problem it critiques.
Results and Conclusion
The project resulted in a functional, portable installation that successfully generates fluent, genre-consistent misinformation. The system can produce text that matches specific styles, such as a "Celebrity Gossip" rumour about a wedding or a "Politics" rumour about the State Department.
The Mill provides a playful yet critical way for users to experience the state of current NLP technology. It moves the conversation from abstract fear of "AI bots" to a concrete understanding of how parameters like genre and randomness influence the "believability" of a lie. While the Mill helps people identify current AI-generated text, the authors acknowledge that as models improve, identifying them will only become more difficult, suggesting a need for ongoing iterations of such educational tools.
Summary updated