TL;DR
This paper introduces ASCENT, a novel methodology and a publicly accessible web portal for building and exploring a deep commonsense knowledge base (CSKB). ASCENT goes beyond traditional triple-based KBs by incorporating semantic facets and composite concepts, offering a richer representation of commonsense knowledge that demonstrably improves question answering performance.
What Problem Does This Paper Address?
The paper addresses the limitations of existing commonsense knowledge bases (CSKBs) and their application in AI tasks, particularly question answering (QA). While large pre-trained language models (LMs) have shown impressive capabilities, they often lack interpretability and explainability, and their implicit commonsense knowledge can be unreliable. Traditional CSKBs, while offering structure, are often limited by a simple subject-predicate-object (SPO) model, failing to capture the nuances of commonsense assertions like temporal or spatial circumstances, causes, or instruments. This limits their expressiveness and, consequently, their effectiveness in complex reasoning tasks.
Methods
ASCENT employs a multi-stage automated pipeline for extracting and consolidating commonsense assertions from the web. Key methodological contributions include:
- Expressive Knowledge Model: ASCENT refines subjects by incorporating subgroups (e.g., "tourist bus") and aspects (e.g., "bus driver"). Crucially, it introduces "semantic facets" to SPO triples, capturing additional context like cause, manner, purpose, location, and temporal information. This moves beyond simple factual statements to richer, more nuanced representations.
- Source Discovery: The pipeline leverages Bing Web Search API with queries refined by WordNet hypernyms to find relevant documents. Documents are then filtered based on similarity to Wikipedia articles.
- Knowledge Extraction: Open Information Extraction (OIE) techniques are used to extract faceted assertions. A supervised classifier labels semantic facets, and subgroups/aspects are identified through noun phrase analysis and clustering.
- Knowledge Consolidation: Assertions are clustered using word2vec and a fine-tuned RoBERTa model to group semantically similar triples and facet values.
- Web Portal: A user-friendly web portal is provided for exploring the ASCENT KB, visualizing its construction process, and experimenting with its use in QA.
Datasets
The paper does not explicitly list specific datasets used for training the ASCENT pipeline itself, but it mentions leveraging:
- Web Content: The primary source for extracting commonsense assertions.
- WordNet: Used for disambiguating word senses and refining search queries.
- Wikipedia: Used for filtering relevant documents.
- ConceptNet: Used as a baseline for selecting primary subjects for the ASCENT KB.
- Existing CSKBs (e.g., Cyc, ConceptNet, Atomic, WebChild, TupleKB, Quasimodo, CSKG): These are implicitly used for comparison and evaluation, particularly in the QA experiments.
The paper also makes the ASCENT KB data dump (678MB of 8.9 million assertions) available via the HuggingFace Datasets package.
Key Insights and Ideas
- Beyond SPO: The introduction of semantic facets is a significant step beyond the limitations of traditional SPO triples, allowing for a much richer and more expressive representation of commonsense knowledge. This is crucial for capturing the nuances of real-world reasoning.
- Automated Extraction is Scalable: ASCENT demonstrates that a robust and deep CSKB can be automatically constructed from web data, overcoming the coverage limitations of human-annotated KBs.
- Interpretability Matters: The web portal provides a valuable tool for understanding how commonsense knowledge is extracted and consolidated, fostering trust and enabling deeper analysis of the KB's content.
- CSK Enhances QA: The paper highlights that structured commonsense knowledge can significantly improve the performance of pre-trained language models in question answering tasks, even in zero-shot settings.
Key Results
The paper presents a comprehensive evaluation of the ASCENT KB's contribution to QA. While specific quantitative results for the KB construction itself are not detailed in this demo paper, the authors refer to their technical paper (Nguyen et al., 2021) for details on its salience and recall. The demo paper focuses on the utility of the KB:
- The ASCENT KB contains 8.9 million assertions.
- The web portal allows users to explore the KB, inspect assertion construction, and experiment with QA setups.
- Experiments show that priming LMs with context from CSKBs (including ASCENT) can lead to more accurate and less evasive answers in various QA setups (Masked Prediction, Free Generation, Guided Generation, Span Prediction). For instance, without context, GPT-2 provides an evasive answer to "What do rabbits eat?", but with context from ASCENT, it generates a more informative response.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while ASCENT has better coverage than human-created KBs, it may still contain noise, a common challenge in automated extraction.
- The current implementation of the web portal runs on a virtual machine with limited resources (4 virtual CPUs, 8 GB RAM), which might affect performance for very large-scale explorations.
- Future Work:
- The authors hope the portal will enable interesting interactions with the ASCENT methodology.
- The QA demo allows researchers to explore the potential of combining structured data with pre-trained language models.
- The paper implicitly suggests further exploration of how different QA setups and retrieval strategies interact with structured knowledge.
Summary updated