Advertising Legality Recognition

Yi-jie Tang, Cong-kai Lin, Hsin-Hsi ChenCOLING 2012

TL;DR

This paper tackles the growing problem of illegal online advertising by developing and evaluating machine learning models to automatically detect fraudulent or misleading claims. They introduce a novel feature weighting scheme using log relative frequency ratio that significantly improves classification accuracy, and demonstrate its effectiveness in identifying problematic verb-noun phrases commonly found in illegal ads.

What Problem Does This Paper Address?

The internet has become a dominant advertising platform, leading to a surge in advertisements presented to consumers. This growth has also brought about an increase in false, overstated, or misleading advertisements, creating a critical need for effective methods to identify and flag them. Consumers, advertisers, and regulatory bodies all face challenges in discerning legitimate advertising from illegal content. Existing regulations, particularly in Taiwan for food and cosmetic products, prohibit claims of curative effects or misleading statements. However, manually reviewing the sheer volume of online ads is impractical. This paper aims to bridge this gap by proposing computational approaches to automatically recognize advertising legality.

Methods

The authors explore two primary machine learning classifiers: Naïve Bayes and Support Vector Machines (SVM), implemented using the LIBSVM library. They experiment with various feature sets to represent advertising text:

  • Unigrams: Basic word features, representing the presence of individual words in an advertisement.
  • Unigram + CILIN: Unigrams augmented with words from a Chinese thesaurus related to curative effects.
  • Unigram + DOH: Unigrams enhanced with examples of illegal advertising statements identified by Taiwan's Department of Health (DOH).
  • Unigram + CILIN + DOH: A combination of the above auxiliary features.
  • Unigram + logRF (Log Relative Frequency Ratio): This is the paper's novel contribution. They use the log relative frequency ratio of words between legal and illegal datasets to weight unigram features. This approach highlights words that are significantly more prevalent in one type of advertisement (legal or illegal) compared to the other, effectively capturing characteristic language patterns.

Beyond classification, the paper also introduces a method for illegal verb phrase mining. This involves identifying verb-noun phrases that are disproportionately common in illegal advertisements, using the log relative frequency ratio to pinpoint these problematic expressions.

Datasets

The study utilizes two main types of datasets:

  • Illegal Advertising Data: Collected from official announcements of illegal food and cosmetic advertisements made public by the Taipei City Government between July 2009 and November 2011. These lists include product names and the corresponding problematic advertising statements.
  • Legal Advertising Data: Crawled from an online shopping website in Taiwan. The authors assume these advertisements generally comply with regulations and manually verify them to ensure accuracy.

These datasets are then split into four specific categories for training and testing: FOOD_LEGAL, FOOD_ILLEGAL, COS_LEGAL, and COS_ILLEGAL. The paper notes that all sentences are in Traditional Chinese and undergo word segmentation and part-of-speech tagging.

Key Insights and Ideas

  • Log Relative Frequency Ratio is a Powerful Feature Weight: The most significant insight is that weighting unigram features with the log relative frequency ratio between legal and illegal datasets dramatically improves classification performance. This suggests that the relative prevalence of words is a stronger indicator of legality than their mere presence.
  • Verb-Noun Phrases are Key Indicators of Illegality: The research confirms that many illegal advertisements, particularly in the food and cosmetic domains, rely on specific verb-noun phrases (e.g., "improve physical condition," "cure arrhythmia"). Mining these phrases offers a direct way to identify problematic language.
  • Auxiliary Features Have Limited Impact: While incorporating thesaurus-based (CILIN) and DOH-provided illegal statement examples (DOH) showed some improvement, their impact was less pronounced than the logRF weighting. This suggests that broad thesauri might introduce too much noise, and pre-defined lists, while useful, might not capture the full spectrum of evolving illegal language.
  • SVM Outperforms Naïve Bayes for Legal Ads, but LogRF Levels the Playing Field: SVMs generally performed better for legal advertising detection, while Naïve Bayes was slightly better for illegal ads. However, the Unigram + logRF feature setting allowed both classifiers to achieve high accuracy, with SVM ultimately leading in overall performance.

Key Results

  • Highest Accuracy Achieved with Unigram + logRF: The SVM classifier with the Unigram + logRF feature set achieved the highest accuracies: 93.433% for food and 86.037% for cosmetic datasets.
  • Low False Alarm and Missing Rates: The logRF feature set also yielded low error rates:
  • False Alarm Rates (legal misclassified as illegal): 0.083 (food) and 0.166 (cosmetic).
  • Missing Rates (illegal misclassified as legal): 0.053 (food) and 0.115 (cosmetic).
  • Significant Improvement: The accuracies obtained with Unigram + logRF were statistically significant (p<0.01) compared to other feature settings.
  • Verb Phrase Mining Success: The paper successfully mined a substantial number of illegal verb phrases (979 from FOOD, 2,302 from COS) using the logRF method, providing a valuable resource for identifying problematic language.

Limitations and Future Work

  • Coverage of Auxiliary Features: The authors note that while DOH examples are useful, their coverage might be an issue, and they discuss the need to expand this list.
  • CILIN Thesaurus Noise: The CILIN thesaurus, while intended to help find related words, included many uncommon words that might not be suitable for classification, potentially introducing noise.
  • Future Work: The authors propose extending their methodology to other domains, specifically mentioning the medicine domain, which is also heavily regulated. They also suggest that the developed recognition models and mined verb phrase lists can be used to build automated illegal advertising recognition systems and assist authorities and advertisers.

Summary updated

BibTeX
@inproceedings{tang2012advertising,
  author = {Yi-jie Tang and Cong-kai Lin and Hsin-Hsi Chen},
  title = {{Advertising Legality Recognition}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2012},
}
Figure