Commonsense Knowledge Mining from the Web

Chi-Hsin Yu, Hsin-Hsi ChenAAAI 2010 · DOI

Introduction

A long-standing assumption in artificial intelligence is that "common sense isn't written down" because it is too obvious to mention. This "knowledge acquisition bottleneck" historically forced researchers to choose between expensive expert curation or noisy volunteer-contributed data. This paper challenges this myth by demonstrating that a significant portion of commonsense knowledge (CSK) is explicitly stated on the web. By introducing an automated pipeline to mine this knowledge and a structural classifier, the authors achieve up to 82.6% accuracy in verifying the quality of these assertions.

Methodology

The authors propose a general mining algorithm that utilizes a seed set of knowledge to build a classifier for identifying new patterns on the web. The approach rests on three pillars:

  1. Estimation of Explicitly Stated CSK (e-CSK): Using the Google AJAX Search API, the team measured the frequency of assertions from the Open Mind Common Sense (OMCS) dataset. They developed a linear regression model to estimate full search volumes and employed language models to filter out linguistic noise.
  2. Structural Classification: Moving beyond a simple "Bag-of-Words" (BoW) approach, the researchers developed a representation scheme categorizing words into functional slots: HeadNoun, FirstVerb, and OtherWords (N-V-OW).
  3. Feature Selection and Training: Leveraging the Google Web 1T 5-Gram corpus, word co-occurrences were calculated. The top 10% of features, selected via Pearson correlation, were used to train a Support Vector Machine (SVM) classifier with a radial basis function (RBF) kernel.

Analysis of Web Visibility

The study reveals that the "obvious" is surprisingly popular in writing. For predicates such as HasProperty, over 82% of assertions were found explicitly stated on the web. Furthermore, a clear positive correlation exists between the "confidence score" assigned by human volunteers in OMCS and the frequency of that assertion online.

Visibility of different CSK predicates on the web as a function of confidence score.

While many facts are explicit, some remain "implicit." For instance, while the direct fact "a lobby is used for waiting" may have low search volume, the usage "waiting in the lobby" is ubiquitous. This suggests that while some facts are rarely stated as propositions, their underlying truth is reflected in common linguistic patterns.

Experimental Results

The structural N-V-OW model significantly outperformed the baseline Bag-of-Words approach across various knowledge types. The system achieved a peak accuracy of 82.6% for the HasProperty predicate and 74.4% for IsA relations, which is competitive with specialized semantic relation tasks.

Classification accuracy of the N-V-OW model compared to baseline across various predicates.

The high percentage of support vectors required by the SVM models (>60%) indicates the immense diversity of commonsense knowledge, suggesting it does not cluster into easily separable groups.

Conclusion and Future Work

This research proves that the web is a viable source for mining high-quality commonsense knowledge. However, the breadth of common sense makes a "one-size-fits-all" classifier challenging. Future work should explore domain-specific classifiers and a domain-detection layer to navigate different knowledge areas. Bridging the gap between explicit propositional knowledge and implicit usage-based knowledge remains a critical frontier for building more robust, "sensible" AI systems.

Summary updated

BibTeX
@inproceedings{yu2010commonsense,
  author = {Chi-Hsin Yu and Hsin-Hsi Chen},
  title = {{Commonsense Knowledge Mining from the Web}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2010},
  doi = {10.1609/aaai.v24i1.7505},
}
Figure