Abstract: This research, presented at EMNLP 2016, introduces a significant advancement in Japanese natural language processing by bridging the gap between deep syntactic parsing and formal logical reasoning. The authors present a system that maps wide-coverage Japanese Combinatory Categorial Grammar (CCG) parser outputs to Higher-Order Logic (HOL) representations, enabling automated reasoning on complex linguistic phenomena.
The Challenge of Deep Semantic Reasoning
In computational linguistics, Recognizing Textual Entailment (RTE) is the task of determining if one sentence logically follows from another. While English has mature pipelines for mapping syntax to logic, Japanese has historically relied on "chunk-based" dependency parsing, which is less suited for deep semantic reasoning.
Furthermore, most existing logic-based systems utilize First-Order Logic (FOL). While computationally efficient, FOL struggles to represent "generalized quantifiers" (e.g., most, few) or "intensional modifiers" (e.g., almost, fake). For example, in FOL, it is difficult to represent the fact that "John almost walked" does not entail "John walked." This paper addresses these structural and logical gaps for the Japanese language.
Methodology: From Syntax to Higher-Order Logic
The authors developed a three-stage pipeline to achieve robust semantic inference:
- Syntactic Parsing: The system uses Jigg, a Japanese CCG parser. CCG is a lexicalized grammar that provides a transparent interface for semantic composition.
- Semantic Parsing: The core innovation involves mapping CCG trees onto semantic representations in Higher-Order Logic (HOL) using simply typed $\lambda$-calculus. They adopt Neo-Davidsonian Event Semantics, treating verbs as predicates over events and case markers (like ga and o) as functions mapping events to participants.
- Logical Inference: The resulting HOL formulas are processed by an automated theorem prover to check for entailment or contradiction.
A key technical hurdle was the "NP-structure discrepancy." Unlike English, Japanese is a classifier language where noun phrases (NPs) often appear without determiners. The authors utilized "type-raising" to allow these bare NPs to take modifiers as arguments, ensuring a consistent compositional flow.
Semantic Composition in Action
The power of this approach is most evident when handling complex quantification and modification. By using HOL, the system can treat intensional modifiers as higher-order predicates. This allows for precise derivations of sentences that would be ambiguous or impossible to represent accurately in standard FOL.
Evaluation and Key Results
The system was evaluated on the Japanese Semantics test suite (JSeM), a dataset designed for textual entailment focusing on logical inferences. The evaluation covered 523 problems involving quantifiers, plurals, adjectives, and attitudes.
- High Accuracy: With correct (gold) syntactic trees, the system achieved 86.0% accuracy. In an end-to-end setting using the automated parser, it achieved 75.0% accuracy.
- The Necessity of HOL: An ablation study showed that removing HOL-specific axioms caused accuracy to drop from 86% to 70%, proving that higher-order reasoning is essential for capturing natural language nuances.
- Practicality: Contrary to the belief that HOL is too computationally expensive for real-world NLP, the authors demonstrated efficient inference on complex linguistic structures.
Limitations and Future Directions
While the system is highly effective for structural logical inference, several areas remain for future development. The primary bottleneck is currently syntactic parsing; errors often stem from the CCG parser failing to find the correct structure. Additionally, the current system does not yet handle anaphora resolution or presupposition. Finally, integrating a broad lexical knowledge base (like WordNet) would allow the system to handle inferences based on word meaning (e.g., knowing a "sofa" is "furniture") in addition to logical structure.
Summary updated