Semantic Retrieval for the Accurate Identification of Relational Concepts in Massive Textbases

Yusuke Miyao, Tomoko Ohta, Katsuya Masuda, Yoshimasa Tsuruoka, Kazuhiro Yoshida, Takashi Ninomiya, Jun'ichi TsujiiACL 2006 · OpenReview

Abstract: Yusuke Miyao, Tomoko Ohta, Katsuya Masuda, Yoshimasa Tsuruoka, Kazuhiro Yoshida, Takashi Ninomiya, Jun’ichi Tsujii. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics.

2006.

Introduction

Information retrieval has traditionally been constrained by the "keyword bottleneck." Standard search engines typically identify documents based on word proximity rather than semantic understanding. For instance, a search for "A inhibits B" might return results where "B inhibits A" or where the terms appear unrelatedly within the same text block. This lack of syntactic awareness leads to high noise levels in results, particularly in specialized fields like biomedicine.

Researchers from the University of Tokyo and the University of Manchester addressed this by developing a framework that transitions from simple word matching to a deep understanding of predicate-argument structures—the "who-did-what-to-whom" of a sentence.

The Challenge of Relational Retrieval

In biomedical research, the primary need is often the retrieval of relational concepts, such as specific protein-protein interactions. Traditional Information Retrieval (IR) fails to distinguish between active and passive voice or between positive and negative assertions. A keyword search for "protein X activates gene Y" might incorrectly flag the sentence "The activation of gene Y is not affected by protein X."

Semantic Retrieval aims to solve this by mapping sentences to their underlying logical structures, handling synonyms, and distinguishing between different semantic roles regardless of surface-level phrasing.

Methodology

The proposed framework bifurcates the retrieval process into two distinct phases: off-line pre-processing and on-line query execution.

System architecture for off-line and on-line processing.

Off-line Processing

To handle the scale of massive textbases like MEDLINE (over 15 million entries), the system performs heavy linguistic analysis upfront:

  1. Deep Parsing: Every sentence is analyzed using a Head-driven Phrase Structure Grammar (HPSG) parser to identify predicate-argument structures.
  2. Term Recognition: Ontologies such as GENA and UMLS are used to map technical terms to unique identifiers, ensuring that "CRP" and "C-reactive protein" are treated as the same entity.
  3. Structured Indexing: These annotations are stored using an extended version of Region Algebra, a mathematical framework designed for searching structured text.

On-line Processing

When a user submits a query, it is converted into a formal region algebra query. The search engine matches this pattern against the pre-computed semantic structures, delivering high-precision results in real-time.

Deep Parsing and Semantic Mapping

The core of the system's accuracy lies in its ability to parse complex grammatical structures. By identifying the logical subject and object of every verb, the system can distinguish between different relational directions.

HPSG parse tree and predicate-argument structure.

As shown in the parse tree, the system maps syntactic dependencies to semantic arguments (e.g., ARG1, ARG2). This allows the retrieval engine to understand that in the phrase "does not exclude," the entities involved have specific roles that must match the user's query parameters.

Results and Performance

The system was evaluated against eight complex biomedical queries. Key findings include:

  • Precision: Semantic search significantly outperformed keyword search, often achieving over 90% precision compared to the 10-60% range typical of standard IR.
  • Speed: Despite an index size of nearly 1 Terabyte (954 GB), the system returned relevant answers within seconds, proving the viability of deep linguistic analysis for real-time applications.
  • Relational Accuracy: The system successfully differentiated between inverse relationships (e.g., "A inhibits B" vs. "B inhibits A"), a task impossible for keyword-based systems.

Limitations and Future Work

While the framework represents a significant advancement, several challenges remain:

  • Noun Phrase Complexity: The system occasionally misidentifies relationships within complex noun phrases (e.g., "increased mortality in patients with diabetes").
  • Nominalizations: While effective for verbs, the system is less robust when relationships are expressed as nouns (e.g., "the inhibition of B by A").
  • Coreference Resolution: The current implementation does not track pronouns across sentences, limiting its ability to link entities mentioned via "it" or "they" to their original referents.

This research demonstrates that deep linguistic parsing can be scaled to the size of entire scientific literatures, providing a foundation for modern knowledge-graph-based retrieval systems.

Summary updated

BibTeX
@inproceedings{miyao2006semantic,
  author = {Yusuke Miyao and Tomoko Ohta and Katsuya Masuda and Yoshimasa Tsuruoka and Kazuhiro Yoshida and Takashi Ninomiya and Jun'ichi Tsujii},
  title = {{Semantic Retrieval for the Accurate Identification of Relational Concepts in Massive Textbases}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  url = {https://openreview.net/forum?id=hc7g43EoLtM},
}
Figure