Abstract
While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks. This paper presents a survey of these tasks, discusses the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation as revealed by these tasks, and reflects on future research directions.
Introduction
Commonsense knowledge—the practical, everyday information about how the world works—is something humans take for granted but AI historically finds difficult. Traditionally, researchers addressed this by manually building massive Knowledge Bases (KBs) like Cyc or ConceptNet.
With the rise of the "Neural NLP" era, Pre-trained Language Models (PLMs) like BERT, RoBERTa, and GPT have emerged as potential "soft" alternatives to traditional KBs. This survey investigates the extent to which these models capture commonsense and how effectively they can reason or generate new information. A core challenge identified is "reporting bias": humans rarely document the obvious (e.g., "people breathe air"), meaning models may miss fundamental truths never explicitly stated in their training data.
Methodological Pillars
The paper categorizes research into three main pillars: Knowledge Probing, Commonsense Reasoning, and Commonsense Generation.
1. Knowledge Probing
Knowledge probing uses "clozed" (fill-in-the-blank) templates to test if a model "knows" a fact. This approach treats the PLM as a queryable memory source.
While models can recall factual knowledge reasonably well in zero-shot settings, they struggle with "N-to-M" relations, where one entity relates to many others simultaneously.
2. Commonsense Reasoning
Reasoning is evaluated across six specialized domains: Linguistic, Physical, Abductive, Social, Multimodal, and Temporal.
A significant finding is the "Physical Gap." Models predominantly learn property associations mentioned explicitly in text but lack an understanding of "world dynamics"—the experience of interacting with the physical world. They struggle with state changes and unconventional object uses because they do not truly perceive the physical environment.
3. Commonsense Generation
This pillar tests a model's ability to create coherent text based on concepts (e.g., given {dog, frisbee, catch}) or explain its own reasoning. While models generate grammatically correct sentences, they often suffer from poor coherency, creating scenarios that are linguistically sound but defy commonsense (e.g., "A dog throws a frisbee").
Benchmarks and Performance Realities
While state-of-the-art (SOTA) models reach "near-human" performance on benchmarks like HellaSwag (93.85%) and PIQA (90.13%), the authors argue these numbers may be inflated. Models often exploit "annotation artifacts" or lexical overlaps—shortcuts that allow them to pass tests without true reasoning. For instance, in temporal reasoning, models often rely on shallow pattern cues rather than a deep understanding of time and duration.
Key Insights and Future Directions
- The Power of Multimodality: Models incorporating visual features consistently outperform text-only models, suggesting that visual context is vital for high-quality inference.
- Sensitivity vs. Robustness: PLMs are surprisingly fragile; negating a sentence or adding distracting words can easily flip their predictions.
- Future Work: There is an urgent need for "adversarial filtering" to remove easy shortcuts in datasets. Additionally, research must address reporting bias and expand beyond English-centric datasets to capture cross-lingual commonsense. Modeling human intuition and complex emotional states remains a largely unsolved challenge.
Summary updated