SemNews: A Semantic News Framework

Akshay Java, Tim Finin, Sergei NirenburgAAAI 2006

Abstract

The vast majority of online information remains trapped in natural language text, inaccessible to the sophisticated reasoning capabilities of the Semantic Web. SemNews is a framework designed to bridge this gap by automatically extracting deep semantic meaning from news feeds. By integrating the OntoSem text-processing engine, SemNews converts unstructured RSS summaries into formal OWL/RDF representations. This approach moves beyond shallow Information Extraction, enabling machines to understand complex relationships and events, thereby facilitating advanced querying and reasoning.

Introduction

While the Semantic Web aims to create a web of machine-readable data, traditional Information Extraction (IE) tools often fall short. Most current tools provide "shallow" analysis, such as Named Entity Recognition (NER), which identifies people or places but fails to grasp the underlying relationships or the nature of events. SemNews addresses the challenge of automated semantic annotation at scale. The goal is to ingest news summaries, understand the underlying concepts—such as recognizing a "bombing" as a specific event type with associated actors and locations—and publish this knowledge in a standardized format (OWL) for use by software agents.

Methods

The SemNews framework centers on the integration of OntoSem, a language-independent text analysis environment, into a web-based pipeline. The process involves several distinct stages:

  1. Ingestion: The system monitors live RSS feeds to extract text summaries.
  2. OntoSem Analysis: The text undergoes a multi-stage pipeline involving preprocessing, syntactic parsing, and semantic mapping. As illustrated in the system architecture, this pipeline resolves linguistic ambiguities to produce structured meaning.
The OntoSem text-processing pipeline used in SemNews.
  1. TMR Generation: The output of the analysis is a Text Meaning Representation (TMR), a frame-based internal model of the story's meaning. This maps words to an ontology of approximately 8,000 concepts and a specialized "Onomasticon" of 400,000 proper names.
  2. OntoSem2OWL: A critical component of the framework is the translation of these internal frame-based TMRs and the broader ontology into the Web Ontology Language (OWL).
  3. Storage & Querying: The resulting RDF triples are stored in a Redland triple store. This allows for complex SPARQL/RDQL queries, enabling concept-based retrieval where a search for "terrorist organization" can find relevant stories even without exact keyword matches.

Key Results

The implementation of SemNews demonstrated the feasibility of large-scale semantic translation and querying:

  • Ontology Translation: The researchers successfully translated 7,747 out of 8,000 concepts from the OntoSem ontology into OWL.
  • Knowledge Base Scale: The system generated over 100,000 RDF triples from processed news content.
  • Performance: Building the model using the Jena API takes between 10 to 40 seconds, with basic RDFS inferencing requiring approximately 10 seconds on standard hardware.
  • Query Capability: The framework successfully supports complex semantic queries, such as identifying stories where a specific nation was the site of a specific event type.

Discussion and Future Work

While SemNews demonstrates significant progress in deep semantic annotation, several challenges remain. The authors noted that while "OWL Micro" reasoners performed well, "OWL Full" reasoners struggled with the massive search space created by the large ontology. Furthermore, the translation from frame-based representations to OWL can be "lossy," though it remains sufficient for many web applications.

Future research will focus on formal evaluations of semantic recall and precision. Additionally, the authors plan to utilize semantic TMRs for more advanced news clustering based on actual meaning rather than shared keywords, further enhancing the utility of the SemNews framework.

Summary updated

BibTeX
@inproceedings{java2006semnews,
  author = {Akshay Java and Tim Finin and Sergei Nirenburg},
  title = {{SemNews: A Semantic News Framework}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure