Machine Learning for Information Extraction from XML marked-up text on the Semantic Web

Nigel CollierSemWeb 2001

Abstract

This paper explores the use of Hidden Markov Models (HMMs) to automate the transformation of unstructured text into structured XML data. By leveraging orthographic features and a consistency-focused "Unity" module, the system achieves high accuracy in identifying named entities across disparate domains such as molecular biology and general news, offering a portable alternative to traditional rule-based extraction systems.

Introduction

In the early 2000s, the vision of the "Semantic Web" required computers to understand the underlying meaning of data. However, most information remains trapped in unstructured text. To bridge this gap, Information Extraction (IE) tools are needed to identify specific entities and label them with XML tags. Traditionally, these systems relied on hand-built, rule-based logic, which is notoriously brittle and difficult to port between domains (e.g., moving from news articles to medical journals). This research addresses the portability problem by using machine learning to create a system that adapts to any domain given sufficient marked-up training examples.

Methods

The authors employ a Hidden Markov Model (HMM), a statistical model that predicts a sequence of hidden states (entity labels like "Protein" or "Organization") based on a sequence of observed events (the words in a sentence).

Orthographic Features

To handle "out-of-vocabulary" words not seen during training, the model analyzes the "shape" of a word rather than just the word itself. These orthographic features identify if a word is capitalized, contains digits, or includes specific symbols. This is particularly effective in scientific domains where nomenclature follows specific patterns.

Orthographic feature codes and examples used by the HMM to identify entity types based on word shape.

The "Unity" Module

A key technical contribution is the "Unity" module, a post-processing step designed to ensure label consistency across a document. If the HMM identifies a phrase as a "Protein" in one instance, the Unity module checks other occurrences of that same phrase to ensure the labels match, correcting isolated statistical errors.

Datasets and Evaluation

The model was evaluated on two distinct domains to prove its versatility:

  1. MUC-6 (News): A standard benchmark of 60 news articles annotated for entities like people, organizations, and dates.
  2. Molecular Biology Corpus: 100 PubMed abstracts annotated for complex entities like proteins, DNA, RNA, and cell types. This dataset is significantly more dense and linguistically complex than the news corpus.

Key Results

Performance was measured using the F-score, which balances precision and recall. The inclusion of the Unity module provided a significant boost in both domains, particularly in the news corpus.

F-score performance comparison across domains, demonstrating the impact of the Unity module on extraction accuracy.

While the news scores were slightly higher, the biology results were impressive given the complexity of the terminology. The Unity module proved to be a simple but effective way to catch errors made by the base statistical model, boosting the overall F-score by 2-4%.

Insights and Challenges

The research highlighted that a word's orthographic signature is often more informative than its dictionary definition. For example, a string like [CD4+] has a specific visual pattern (caps, digits, symbols) that strongly signals a cell type even if the specific entity is new to the model.

However, several linguistic "landmines" remain challenging for HMMs:

  • Coordination: Phrases like "v-rel and c-rel proteins" require the model to associate "v-rel" with the distant word "proteins."
  • Apposition: Identifying relationships in phrases like "Transcription factor NF-kappa B."
  • Abbreviations: Linking acronyms to their long-form versions within the same text.

Conclusion

While HMMs are limited by their focus on local context, they represent a massive leap forward for the Semantic Web by enabling the automated extraction of flat entity sequences. Future work suggests moving toward Support Vector Machines (SVMs) and incorporating shallow dependency analysis to handle more complex linguistic structures.

Summary updated

BibTeX
@inproceedings{collier2001machine,
  author = {Nigel Collier},
  title = {{Machine Learning for Information Extraction from XML marked-up text on the Semantic Web}},
  booktitle = {SemWeb},
  year = {2001},
}
Figure