Abstract
Commonsense Knowledge Graphs (CSKGs) like ATOMIC are vital for social reasoning but suffer from structural sparsity, often existing as isolated bipartite "islands" that hinder multi-hop reasoning. This paper introduces Dense-ATOMIC, a significantly more connected version of the original graph. By normalizing event descriptions and employing a novel relation-prediction model (Rel-CSKGC) based on RoBERTa-large, the researchers increased one-hop connections by approximately 3x and multi-hop paths by several orders of magnitude. Evaluation shows high precision (0.80) and improved performance in downstream commonsense generation tasks.
Introduction: The Sparsity Problem in ATOMIC
Teaching AI everyday social logic—such as understanding that "If X asks Y to marry, X likely loves Y"—requires robust Commonsense Knowledge Graphs. ATOMIC has been a cornerstone in this field, yet its construction via "one-hop" annotations created a structural bottleneck. Annotators typically linked a base event to a tail event, resulting in a collection of independent bipartite graphs. In this architecture, tail events rarely connect back to other base events or to each other, leading to low knowledge coverage and a lack of multi-hop paths necessary for complex reasoning chains (e.g., Event A $\rightarrow$ B $\rightarrow$ C).
Methodology: Densifying the Graph
To address these limitations, the authors developed a three-stage pipeline to "densify" the ATOMIC graph, transforming it from a collection of fragments into a cohesive network.
- Event Normalization: In the original ATOMIC, base events are full sentences, while tail events are often fragments (e.g., "loving"). Using NLP tools like CoreNLP and WordNet, the authors normalized all tail events into a consistent "Subject + Verb + Object" pattern (e.g., "X is loving"). This allows any node to serve as either a head or a tail, facilitating new connections.
- Rel-CSKGC (Relation Prediction): Traditional knowledge graph completion models focus on predicting a missing tail. However, in sparse graphs, this is inefficient. The authors instead built Rel-CSKGC, a model that takes two existing events and predicts the most likely relation between them using a RoBERTa-large backbone.
- Intra-and-Inter Cluster Completion: To manage computational complexity, the researchers employed a sampling strategy. They checked for missing links within "clusters" (events derived from the same base) and between different clusters, effectively bridging the "islands" of knowledge.
Results and Discussion
The transformation from ATOMIC to Dense-ATOMIC yielded dramatic quantitative improvements. The number of one-hop paths grew to approximately 1.9 million, nearly triple the original count. More impressively, the connectivity explosion unlocked massive multi-hop reasoning potential.
As shown in Figure 1, the dense version fills in missing logical links (e.g., connecting "X asks Y to marry" to "Y is happy" via predicted relations). The statistical impact is staggering:
- 2-hop paths increased by ~550x (from 19k to 10.6 million).
- 3-hop paths saw a ~130,000x increase (from 509 to over 67 million).
Human evaluation confirmed the quality of these new links, with the Rel-CSKGC model achieving a precision of 0.80. Furthermore, models like COMET trained on Dense-ATOMIC demonstrated superior diversity and quality in commonsense generation compared to those trained on the original sparse graph.
Conclusion and Future Work
Dense-ATOMIC represents a significant step forward in building usable commonsense resources. By shifting the focus from tail prediction to relation prediction and normalizing event structures, the authors have unlocked a wealth of latent knowledge within ATOMIC. While the current model focuses on the "most reasonable" relations, future work will explore multi-relational links and scaling the completion process to the entire graph to further enhance reasoning capacity.
Summary updated