Context Analysis System for Japanese Text

Hitoshi Isahara, Shun IshizakiCOLING 1986

TL;DR

This paper introduces a novel system for understanding Japanese text by integrating syntactic, semantic, and contextual analysis. It tackles the challenge of extracting deeper meaning beyond literal word-for-word translation, particularly for complex topics like terrorism, by building a structured contextual representation.

What Problem Does This Paper Address?

While significant progress has been made in syntactic analysis of natural language, understanding the deeper contextual meaning remains a significant challenge. Humans effortlessly grasp nuances, inferring information and understanding the "gist" of a text. This paper aims to bridge this gap by developing a system that can move beyond surface-level parsing to extract contextual information, specifically focusing on understanding texts related to terrorism. The authors highlight that existing systems often struggle with the ambiguity and implicit information present in natural language, especially in languages like Japanese with complex grammatical structures.

Methods

The system employs a serial, integrated approach comprising three main analyzers:

  • Syntactic Analyzer: This component uses a context-free grammar parser (Extended-Lingol) to generate parsing trees from input Japanese sentences.
  • Semantic Analyzer: This layer builds word-level semantic structures, represented in a frame-like format. It focuses on modifying relations within noun phrases and case relations between verbs and noun phrases. The system utilizes a semantic representation language (SRL) that allows for deep semantic analysis and hierarchical concept representation.
  • Contextual Analyzer: This is the core of the paper's contribution. It takes the semantic structures and uses a pre-defined contextual knowledge structure (a tree structure representing scenes and their relationships) to extract contextual information. The system uses a concept thesaurus to flexibly match input events to scenes within this structure.

Two methods for selecting the appropriate contextual structure are proposed:

  • Two-Event Method: Selects a structure based on identifying two key events within the input text.
  • Title-Based Method: Leverages the title of newspaper articles to directly select a relevant contextual structure, which the authors found to be more effective and similar to human behavior.

Datasets

The paper uses newspaper articles as its primary data source. A specific example of a Japanese newspaper article about a bombing incident in Sicily and its English translation is provided to illustrate the system's functionality. No specific training or evaluation datasets are explicitly listed, but the system is demonstrated on an example article.

Key Insights and Ideas

  • Serial Integration of Analysis: The paper argues for a serial, rather than simultaneous, integration of syntactic, semantic, and contextual analysis for computational efficiency, even though humans might process these simultaneously.
  • Contextual Knowledge Structures: The core idea is to represent contextual knowledge as a hierarchical tree structure of "scenes" that can be instantiated with events from the input text. This allows the system to understand the relationships between events and infer missing information.
  • Flexibility through Concept Thesaurus: The use of a concept thesaurus enables flexible matching between input events and scenes in the contextual structure, allowing for the identification of higher-level concepts (e.g., "murder" as a type of "damage").
  • Title-Based Selection: Newspaper titles are identified as a rich source of information for selecting the correct contextual structure, mirroring human intuition.

Key Results

The paper demonstrates the system's ability to process a Japanese newspaper article about a bombing incident and extract contextual information. The "title-based method" for contextual structure selection is shown to be effective, with a high success rate in topic selection for various categories of newspaper articles (e.g., Kidnapping, Explosion, Shooting). The system successfully identifies events like "explosion," "murder," and "injury" and maps them to the appropriate scenes within the "terrorism involving bomb" contextual structure.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Scope: The current system is primarily focused on understanding texts related to terrorism.
  • Dictionary Size: The system's dictionary is still relatively small.
  • Development Stage: The system is still in its development stage, with some parts incomplete.

Future work includes:

  • Increasing Contextual Structures: Expanding the number of available contextual structures to cover a wider range of topics.
  • Enlarging Concept Thesaurus: Growing the scale of the concept thesaurus for more comprehensive understanding.
  • Integrating Processes: Combining the three analysis processes (syntactic, semantic, contextual) into a single, more human-like process.
  • Computer Facilities: Considering the impact of computer facilities on system performance and scalability.

Summary updated

BibTeX
@inproceedings{isahara1986context,
  author = {Hitoshi Isahara and Shun Ishizaki},
  title = {{Context Analysis System for Japanese Text}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1986},
}

Loading related content...

Figure