TL;DR
"The Rumour Mill" is a physical art installation that uses state-of-the-art language models (GPT-2 and CTRL) to turn the abstract threat of AI-generated misinformation into a tangible, interactive experience. By allowing users to "crank out" fake news using physical dials for "wackiness" and "genre," the project makes the covert process of misinformation explicit and educational.
What Problem Does This Paper Address?
As AI-based language models become more sophisticated, they have reached a point where automatically generated text is often indistinguishable from human-written content. This creates a massive scalability problem for misinformation: bad actors can generate believable, high-volume fake news at a very low cost, while the effort required for human fact-checkers to verify and debunk these claims remains high (a "Sisyphean" task).
The authors argue that this process is usually "covert"—happening behind closed doors or via anonymous bot farms. There is a lack of public understanding regarding how these models work and what they are capable of. The "Rumour Mill" addresses this by bringing the generation process into the physical world, making it overt, visible, and even "awkward" to operate, thereby demystifying the technology.
Methods
The authors built a tabletop interactive machine housed in a wooden and plexiglass box. The system uses a two-step natural language generation (NLG) pipeline:
- Headline Generation: The system uses a GPT-2 instance fine-tuned on a dataset of headlines from fact-checking sites. This ensures the output mimics the specific "hook" style of viral misinformation.
- Body Text Generation: The generated headline is fed as a seed into CTRL (Conditional Transformer Language Model). CTRL is specifically designed for "controllable" generation, allowing the system to influence the output based on three physical user inputs:
- Wackiness: A potentiometer maps to the model's "temperature" setting. Low wackiness produces safe, high-probability word choices; high wackiness leads to unusual, unpredictable, and "creative" phrasing.
- Genre: A 12-step switch allows users to select the "flavor" of the rumour (e.g., Conspiracy Theory, CNN Business, Celebrity Gossip, or Russia Today).
- When: A toggle switch sets the time period (Past, Present, or Future) by injecting specific date-related control codes into the model.
The final product is printed on a thermal receipt printer, complete with metadata about the parameters used to create it.
Datasets
- For GPT-2 Fine-tuning: The authors used titles and claims from Snopes, Politifact, and Emergent. This specialized dataset teaches the model the linguistic patterns of debunked rumours and "clickbait" style claims.
- For CTRL: The project leverages the pre-trained CTRL model, which was originally trained on a massive 140GB corpus including Wikipedia, OpenWebText, and various news domains. The authors utilize CTRL's built-in "control codes" to steer the generation toward specific genres.
Key Insights and Ideas
- Tangibility as a Deterrent: By making the machine large and the process of "milling" a rumour a physical act (turning a rotary lever), the authors transform a digital threat into a social performance. It’s hard to spread a rumour "covertly" when you are physically cranking a noisy machine in a public space.
- Explaining "Temperature" through "Wackiness": One of the most elegant translations in the project is mapping the technical concept of "stochastic sampling temperature" to a physical dial labeled "wackiness." It provides a non-technical audience with an intuitive grasp of how AI randomness works.
- Ethical Labeling: To prevent the machine from becoming an actual source of harm, every printout is clearly labeled as a generated rumour. This follows "critical design" principles—creating an object that provokes thought without causing the very problem it critiques.
- The Asymmetry of Information: The paper highlights the "renewable, tireless" nature of AI text generation vs. the "high effort" of human verification. The Mill serves as a physical manifestation of this imbalance.
Key Results
The project resulted in a functional, portable installation that successfully generates fluent, genre-consistent misinformation.
- Qualitative Success: The system can produce text that matches specific styles (e.g., a "Celebrity Gossip" rumour about a wedding vs. a "Politics" rumour about the State Department).
- Public Engagement: The Mill provides a "playful" yet "critical" way for users to experience the state of current NLP technology. It moves the conversation from abstract fear of "AI bots" to a concrete understanding of how parameters like genre and randomness influence the "believability" of a lie.
Limitations and Future Work
- Computational Requirements: Generating high-quality text in real-time is resource-intensive. The current Mill relies on a remote server to handle the heavy lifting, though it includes a local cache to handle temporary network drops.
- Evaluation: As an "Extended Abstract" for a demo, the paper does not include a formal user study or quantitative metrics on how much the installation actually improves "misinformation literacy" among participants.
- The "Arms Race": While the Mill helps people identify current AI-generated text, the authors acknowledge that as models improve, identifying them will only become more difficult, suggesting a need for ongoing iterations of such educational tools.
Summary updated