Abstract
Social Commonsense Reasoning requires understanding of text, knowledge about social events and their pragmatic implications, as well as commonsense reasoning skills. In this work we propose a novel multi-head knowledge attention model that encodes semi-structured commonsense inference rules and learns to incorporate them in a transformer-based reasoning cell. We assess the model's performance on two tasks that require different reasoning skills: Abductive Natural Language Inference and Counterfactual Invariance Prediction as a new task. We show that our proposed model improves performance over strong state-of-the-art models (i.e., RoBERTa) across both reasoning tasks. Notably we are, to the best of our knowledge, the first to demonstrate that a model that learns to perform counterfactual reasoning helps predicting the best explanation in an abductive reasoning task. We validate the robustness of the model's reasoning capabilities by perturbing the knowledge and provide qualitative analysis on the model's knowledge incorporation capabilities.
Introduction
While large language models (LMs) like RoBERTa excel at capturing statistical patterns, they often struggle with "social commonsense"—the implicit understanding of human motivations, emotions, and subsequent actions. This paper addresses two primary bottlenecks in AI: knowledge acquisition (retrieving relevant social knowledge) and knowledge integration (effectively injecting this knowledge into neural networks for multi-fact reasoning).
To evaluate these capabilities, the authors focus on two tasks:
- $α$NLI (Abductive Natural Language Inference): Choosing the most plausible explanation between two observations ($O_1$ and $O_2$).
- CIP (Counterfactual Invariance Prediction): A new task where the model must determine if a factual outcome remains true given a counterfactual alternative to a preceding event.
Methods: The MHKA Model
The proposed Multi-Head Knowledge Attention (MHKA) model utilizes a three-stage pipeline to bridge the gap between text and reasoning. First, Semantic Extraction uses Semantic Role Labeling (SRL) to identify core event components. Second, Dynamic Knowledge Generation employs COMET 2.0 (a generative transformer) to produce context-specific social dimensions (e.g., intent, reactions, wants) from the ATOMIC knowledge graph.
The core architecture consists of three primary layers:
- Context Encoding Layer: Uses RoBERTa to represent input text.
- Knowledge Encoding Layer: Employs transformer blocks to encode generated commonsense rules.
- Reasoning Cell: A novel component using multi-head attention to allow the context to attend to multiple pieces of knowledge simultaneously, enabling the model to "compose" different rules to reach a conclusion.
Key Insights and Results
The authors demonstrate that social reasoning is inherently compositional. By allowing the model to attend to different "heads" of knowledge (intent, need, effect) simultaneously, it can better simulate human-like logic. A significant finding is the link between abduction and counterfactuals: pre-training on counterfactual reasoning tasks directly improves performance on abductive reasoning.
Performance Highlights: * State-of-the-Art: MHKA built on RoBERTa-Large achieved 87.12% accuracy on $α$NLI, significantly outperforming the standard RoBERTa-L baseline (84.48%). * Low-Resource Efficiency: Gains were particularly notable in low-resource settings (1-10% of data), suggesting that explicit knowledge acts as a powerful regularizer when data is scarce. * Transfer Learning: Fine-tuning on $α$NLI after CIP training provided a +3.97 percentage point boost, confirming counterfactual reasoning as a strong pre-training signal.
Robustness tests involving knowledge perturbation confirmed that the model relies on relevant commonsense rules rather than memorizing noise.
Conclusion
The MHKA model represents a significant step toward integrating structured knowledge into neural reasoning. While a performance gap remains between current models and human benchmarks (particularly in the CIP task), the success of multi-head knowledge attention and the synergy between abductive and counterfactual reasoning provide a promising path forward. Future work will explore more complex, compositional knowledge representations and the interaction between social rules and general encyclopedic knowledge.
Summary updated