How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements

Chen Shani, Nadav Borenstein, Dafna Shahaf

Abstract

Humor is an important social phenomenon, serving complex social and psychological functions. However, despite being studied for millennia humor is computationally not well understood, often considered an AI-complete problem. In this work, we introduce a novel setting in humor mining: automatically detecting funny and unusual scientific papers. We are inspired by the Ig Nobel prize, a satirical prize awarded annually to celebrate funny scientific achievements (example past winner: "Are cows more likely to lie down the longer they stand?"). This challenging task has unique characteristics that make it particularly suitable for automatic learning. We construct a dataset containing thousands of funny papers and use it to learn classifiers, combining findings from psychology and linguistics with recent advances in NLP. We use our models to identify potentially funny papers in a large dataset of over 630,000 articles. The results demonstrate the potential of our methods, and more broadly the utility of integrating state-of-the-art NLP methods with insights from more traditional disciplines.

Introduction

Humor is often described as an "AI-complete" problem because it requires a deep cocktail of common sense, empathy, linguistic nuance, and cultural context. While AI has made strides in recognizing puns or one-liners, "scientific humor" is a different beast. It is sophisticated, niche, and often relies on the subversion of academic norms.

This research aims to bridge the gap between computational linguistics and the "Ig Nobel Prize"—a satirical award for achievements that "first make people laugh, and then think." Identifying these papers is more than a novelty; it provides a window into scientific creativity. Quirky research often represents pioneering attempts to expand the frontiers of science, such as the 2000 Ig Nobel winner who levitated a frog with magnets and later won a real Nobel Prize for graphene.

Methodology

The researchers employed a two-pronged approach, comparing "expert-driven" feature engineering with "data-driven" deep learning:

  1. Feature Engineering (The "Iggy" Model): The team formalized 127 features based on four pillars of humor theory: Unexpectedness (measuring perplexity via N-grams, GPT-2, and BERT), Simplicity (title length, readability, and Age of Acquisition), Crude Language (scatological or toxic content), and Funny Language (inherent funniness of specific nouns).
  2. Deep Learning Baselines: The study utilized vanilla BERT and SciBERT (a version of BERT pre-trained on scientific corpora).
  3. Hybrid Models ($BERT^f$ and $SciBERT^f$): The most advanced approach concatenated the 127 "humor features" with deep learning embeddings to leverage both linguistic theory and raw computational power.

Datasets

The models were trained on a "Funny" set of 1,707 titles (including Ig Nobel winners and crowdsourced quirky science) and a balanced "Serious" set of 1,707 titles from Semantic Scholar. Finally, a "Wild" corpus of over 630,000 articles was used to evaluate real-world retrieval performance.

Key Results and Analysis

The hybrid models demonstrated superior performance across classification tasks. Specifically, the $SciBERT^f$ model achieved the highest accuracy at 92.2% on the labeled dataset, significantly outperforming standard baselines.

Classification performance of various models on the labeled dataset, showing that the hybrid SciBERT model achieves the highest accuracy and recall.

Retrieval in the Wild

In a large-scale test ranking 630,000 papers, the models were tasked with finding "needles in a haystack." The Iggy model achieved a Precision@50 of 0.6, meaning 60% of its top 50 recommendations were verified as "funny" by human annotators. This represents a massive leap over the random baseline of approximately 3%.

Precision at K for identifying funny papers in a large-scale retrieval task, comparing various models against a random baseline.

Key Insights

  • The Title is Enough: For scientific humor, the title is usually a sufficient proxy for the content. The "punchline" of a quirky paper is often its premise (e.g., "Chickens prefer beautiful humans").
  • Simple + Unexpected = Funny: The most predictive features combined simplicity with low probability. A funny title uses everyday words in a context where they are statistically unexpected.
  • Model Efficiency: The "Iggy" model, with only 33k parameters, performed remarkably well in retrieval tasks compared to BERT (110 million parameters), suggesting that domain-specific linguistic theories still hold significant weight.

Limitations and Future Work

Humor remains inherently subjective; what one researcher finds "quirky," another might find "serious" or "disturbing." Current models occasionally flag papers about sensitive topics because they use "unexpected" language, highlighting a need for better sentiment filtering. Future directions include expanding analysis to full paper texts and exploring multi-task learning to understand the intersection of humor, creativity, and scientific impact.

Summary updated

Abstract

Humor is an important social phenomenon, serving complex social and psychological functions. However, despite being studied for millennia humor is computationally not well understood, often considered an AI-complete problem. In this work, we introduce a novel setting in humor mining: automatically detecting funny and unusual scientific papers. We are inspired by the Ig Nobel prize, a satirical prize awarded annually to celebrate funny scientific achievements (example past winner: "Are cows more likely to lie down the longer they stand?"). This challenging task has unique characteristics that make it particularly suitable for automatic learning. We construct a dataset containing thousands of funny papers and use it to learn classifiers, combining findings from psychology and linguistics with recent advances in NLP. We use our models to identify potentially funny papers in a large dataset of over 630,000 articles. The results demonstrate the potential of our methods, and more broadly the utility of integrating state-of-the-art NLP methods with insights from more traditional disciplines.
BibTeX
@inproceedings{shani2021how,
  author = {Chen Shani and Nadav Borenstein and Dafna Shahaf},
  title = {{How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=jkrzgRta0T},
}
Figure