Acquiring Comparative Commonsense Knowledge from the Web

Niket Tandon, Gerard de Melo, Gerhard WeikumAAAI 2014 · OpenReview

Abstract

Applications are increasingly expected to make smart decisions based on what humans consider basic commonsense. An often overlooked but essential form of commonsense involves comparisons, e.g. the fact that bears are typically more dangerous than dogs, that tables are heavier than chairs, or that ice is colder than water. In this paper, we first rely on open information extraction methods to obtain large amounts of comparisons from the Web. We then develop a joint optimization model for cleaning and disambiguating this knowledge with respect to WordNet. This model relies on integer linear programming and semantic coherence scores. Experiments show that our model outperforms strong baselines and allows us to obtain a large knowledge base of disambiguated commonsense assertions.

Introduction

While modern AI excels at retrieving factual data (e.g., "What is the capital of Brazil?"), it often struggles with "commonsense"—the foundational knowledge humans use for daily reasoning. A critical subset of this is comparative knowledge. For instance, a digital assistant recommending a "healthier" meal needs to understand that seafood is generally healthier than fast food. Existing structured knowledge bases like Wikipedia or Freebase are rich in static facts but lack these relative tendencies. Furthermore, extracting this information from the web often results in ambiguous text strings. Does "hotter" refer to temperature or physical attractiveness? To be useful for machine reasoning, these terms must be disambiguated into specific semantic "senses."

Methodology

The researchers developed a multi-stage pipeline to extract and refine over one million comparative assertions from the ClueWeb09 and ClueWeb12 corpora, totaling over 1.2 billion web pages (roughly 27 TB of data).

Extraction and Grounding

The process begins with Open Information Extraction (Open IE) using pattern-matching to identify triples in the format (noun phrase) + (comparative predicate) + (noun phrase). Following extraction, Candidate Grounding maps these phrases to potential senses in WordNet. This includes handling "ad hoc" concepts—complex phrases like "private school"—by disambiguating the head noun while preserving the modifier.

Examples of extracted comparative triples across different categories, showing disambiguated WordNet senses.

The Joint Optimization Model

The core innovation is a Joint Model powered by Integer Linear Programming (ILP). Unlike "local" models that disambiguate words in isolation, the Joint Model optimizes for global coherence across the entire knowledge base. It rewards internal coherence (senses within a triple should be related), cross-triple coherence (consistent sense assignment for the same word across different comparisons), and semantic equivalence (using synonyms and antonyms to link related triples).

Results and Evaluation

The system was evaluated against three gold-standard test sets representing standard WordNet entries and ad hoc concepts. The Joint Model demonstrated a significant performance leap over traditional methods.

Performance comparison (precision) between the Most Frequent Sense (MFS) baseline, a Local Model, and the proposed Joint Model.

As shown in the results, the Joint Model achieved a precision of 83%, effectively doubling the performance of the "Most Frequent Sense" (MFS) baseline (~43%) and significantly outperforming a "Local Model" (~47%) that only considered internal triple coherence. The final output is a semantically refined knowledge base containing approximately one million disambiguated assertions.

Conclusion

This research demonstrates that comparative commonsense can be harvested at scale from the web by treating disambiguation as a global optimization problem. While the system currently relies on WordNet's lexical structure and faces challenges with directional noise in web text, it provides a massive, structured foundation for future commonsense reasoning engines. Future work may explore how these millions of triples can be integrated into complex, multi-step reasoning tasks.

Summary updated

Abstract

Applications are increasingly expected to make smart decisions based on what humans consider basic commonsense. An often overlooked but essential form of commonsense involves comparisons, e.g. the fact that bears are typically more dangerous than dogs, that tables are heavier than chairs, or that ice is colder than water. In this paper, we first rely on open information extraction methods to obtain large amounts of comparisons from the Web. We then develop a joint optimization model for cleaning and disambiguating this knowledge with respect to WordNet. This model relies on integer linear programming and semantic coherence scores. Experiments show that our model outperforms strong baselines and allows us to obtain a large knowledge base of disambiguated commonsense assertions.
BibTeX
@inproceedings{tandon2014acquiring,
  author = {Niket Tandon and Gerard de Melo and Gerhard Weikum},
  title = {{Acquiring Comparative Commonsense Knowledge from the Web}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2014},
  url = {https://openreview.net/forum?id=rkZToJWdWB},
}
Figure