"8 Amazing Secrets for Getting More Clicks": Detecting Clickbaits in News Streams Using Article Informality

Prakhar Biyani, Kostas Tsioutsiouliklis, John BlackmerAAAI 2016 · DOI

Introduction

News aggregators like Yahoo News and Google News rely heavily on Click-Through Rate (CTR) to rank content. Clickbait exploits this by using "curiosity gaps," exaggeration, and inflammatory language to bait users into clicking. However, once the user arrives, the content is often low-quality, leading to a frustrating user experience and high abandonment rates.

Examples of clickbait headlines found in news streams.

The challenge is that clickbait isn't "spam" or "fake news" in the traditional sense; it often lives on reputable sites and uses valid English. Because clickbait naturally has high CTR, standard recommendation algorithms actually end up promoting it. This paper seeks to build a classifier that can detect clickbait automatically so it can be demoted or blocked, regardless of how many clicks it attracts.

Methodology

The authors framed the problem as a binary classification task (clickbait vs. non-clickbait) using Gradient Boosted Decision Trees (GBDT). Their innovation lies in the feature set, which goes beyond the headline to analyze the "informality" of the landing page.

Feature Engineering

  • Informality Metrics: They used established readability scores like the Coleman-Liau Score, RIX, and LIX indices, along with a Formality Measure (F-measure). These quantify how "professional" or "casual" the writing is.
  • Forward-Reference Features: They tracked grammatical cues that create a tease, such as demonstrative pronouns ("This is the most horrifying story") or starting a title with an adverb.
  • Similarity Features: They measured the TF-IDF similarity between the headline and the first few sentences of the article body. The intuition is that clickbait headlines often "promise" something the body text doesn't immediately deliver.
  • Content & URL Features: Standard NLP features like n-grams, punctuation counts, and URL structure.

Datasets and Categorization

The researchers curated a custom dataset from the Yahoo homepage stream, including articles from diverse sources like the Huffington Post, New York Times, and Forbes. The dataset comprised 1,349 clickbait and 2,724 non-clickbait web pages collected over a one-year period (2014-2015).

To ensure rigorous labeling, the authors formalized eight distinct types of clickbait, ranging from exaggeration to "bait-and-switch" tactics.

Taxonomy of clickbait types used for dataset labeling, including definitions and examples.

Key Insights

  • Informality is the Smoking Gun: The single most important feature for detection was the F-measure (formality). Clickbait tends to use less formal language than professional news.
  • The Curiosity Gap is Quantifiable: By tracking "forward references" (words like this, he, she in headlines that refer to something only revealed in the body), the model captures the linguistic "itch" that clickbait tries to make you scratch.
  • Headline vs. Body Dissonance: Clickbaits often have lower textual similarity between the title and the actual content compared to genuine news, confirming the "bait-and-switch" nature of the genre.

Results

The model achieved a 74.9% weighted F-1 score on a held-out test set. For the clickbait class specifically, the model showed a precision of 0.717 and a recall of 0.52. This suggests the model is quite accurate when it flags something as clickbait, though it doesn't catch every single instance.

Performance varied by category: the model was most effective at catching Formatting (61.17% recall) and Exaggeration (61.18% recall) but struggled with Graphic content (40.57% recall), likely because those rely more on images than text.

Limitations and Future Work

The current model is purely text-based. The authors note that "Graphic" clickbait often uses shocking images or videos that their model cannot "see." Integrating computer vision features is a suggested next step. Additionally, articles that are factually incorrect but written in a formal, journalistic style (fake news) remain difficult to catch because they don't exhibit the "informality" signal. Future iterations might incorporate user signals, such as comments or social media engagement patterns, to further improve detection accuracy.

Summary updated

BibTeX
@inproceedings{biyani2016amazing,
  author = {Prakhar Biyani and Kostas Tsioutsiouliklis and John Blackmer},
  title = {{"8 Amazing Secrets for Getting More Clicks": Detecting Clickbaits in News Streams Using Article Informality}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2016},
  doi = {10.1609/aaai.v30i1.9966},
}
Figure