TL;DR
This paper introduces two novel systems that leverage analogical inference to automatically generate new common sense facts from a large, noisy database (Open Mind Common Sense). By treating common sense knowledge as a graph of concepts and relations, these systems can infer plausible new statements, significantly accelerating the growth of common sense knowledge bases.
What Problem Does This Paper Address?
Building comprehensive common sense knowledge bases is a monumental task. Traditional methods often rely on manual curation, which is slow and expensive. While analogical reasoning has been explored for common sense, the availability of large, publicly accessible common sense databases has only recently made it feasible to apply these techniques at scale. This paper addresses the challenge of efficiently and automatically augmenting these large, noisy common sense databases by developing systems that can infer new, plausible facts through analogy. The goal is to create a feedback loop where user-contributed facts can be rapidly expanded with system-generated deductions, which users can then validate, further enriching the database.
Methods
The paper presents two distinct analogical inference systems:
- Analogy over Concepts and Relations: This system takes an input sentence and identifies its constituent concepts and relations. It then searches the database for other relations involving these concepts. By substituting different sets of concepts connected by these relations into the original relation structure, it generates new, analogous statements. For example, if the system knows "A ? can have a ?" and finds that "girl" and "small dog" are connected by "A ? will feed her ?", it can infer "A girl can have a small dog."
- Analogies as Inference Rules: This system first models the common sense database as a graph where nodes are concepts and edges are relations. It then identifies "cycles" of three 2-part relations connecting three concepts. These cycles are treated as inference rules. When a new sentence is input, the system attempts to match parts of its relation with these pre-defined inference rules. By instantiating the rule with the input sentence's concepts, it can deduce new statements. For instance, if a cycle implies "If A is like B and C is a B, then A is a C," and the input is "Bats like darkness," it might infer new facts if it finds other relations involving "darkness" and other concepts.
Datasets
The primary dataset used is the Open Mind Common Sense (OMCS) database. This database contains "several hundred thousand English common sense statements" contributed by internet users. The paper does not introduce new benchmarks but uses random sentences from the OMCS database for evaluation.
Key Insights and Ideas
- Leveraging Large, Noisy Databases: The work demonstrates that analogical inference can be effectively applied to large, real-world common sense databases, which are inherently noisy and incomplete, not just curated toy examples.
- Automated Knowledge Augmentation: The core idea is to automate the expansion of common sense knowledge bases. By generating plausible deductions, the systems can significantly speed up the data collection process, allowing users to validate or reject these inferences.
- Two Complementary Analogical Approaches: The paper showcases two distinct but complementary methods for analogical inference, one focusing on concept/relation substitution and the other on rule generation from cyclical patterns.
- Potential for "New Ideas": The systems are capable of generating statements that are not explicitly present in the original database, effectively creating "new ideas" or plausible hypotheses.
Key Results
- System Performance:
- "Analogy over concepts and relations" produced output for 55% of tested sentences.
- "Analogies as inference rules" produced output for 67.5% of tested sentences.
- Accuracy of Inferences:
- For "analogy over concepts and relations," an average of 66% of its new, true statements were generated.
- For "analogy over relations," an average of 64.8% of its new, true statements were generated.
- For "analogies as inference rules," an average of 35.6% of its new, true statements were generated.
- Speed: Both systems typically run within a second on an 800 MHz machine with 512 MB RAM, though "analogies as inference rules" can take several seconds for certain inputs.
- Qualitative Examples: The paper provides examples like deducing "A girl can have a small dog" from "A mother can have a baby" and inferring "If you want to read a newspaper then you should have eyes" from "Seeing requires that your eyes are open."
Limitations and Future Work
- Meaningfulness of Analogies: The paper acknowledges that "sometimes the systems will make analogies that do not fit well."
- Improving Meaningfulness: The probability of meaningful output can be increased by incorporating "negative expertise, probability, and plausibility."
- Integration with Data Collection: The authors highlight that this work is being integrated into OMCS's data collection mechanism to facilitate user validation of generated facts.
- Future Directions: The authors suggest that incorporating mechanisms for negative expertise, probability, and plausibility would enhance the quality of the generated inferences.
Summary updated