Extracting Information about Outbreaks of Infectious Epidemics

Roman Yangarber, Lauri Jokipii, Antti Rauramo, Silja HuttunenEMNLP 2005

TL;DR

This paper introduces ProMED-PLUS, a system that automatically extracts information about infectious disease outbreaks from plain-text reports. It addresses the challenge of processing vast amounts of unstructured data in real-time to build a comprehensive, publicly accessible epidemiological database.

What Problem Does This Paper Address?

The paper tackles the problem of efficiently and accurately extracting critical information from the deluge of text-based reports about infectious disease outbreaks worldwide. Traditional Information Extraction (IE) systems often operate on small, carefully annotated datasets ("closed experiments"). However, the real-world scenario of tracking epidemics involves a dynamic, massive, and ever-growing collection of unstructured text. Manually processing this information is infeasible, leading to delays in understanding and responding to outbreaks. The authors aim to bridge this gap by developing a large-scale, automated IE system that can handle real-time data and provide a valuable resource for researchers.

Methods

The ProMED-PLUS system employs a pipeline architecture for its core IE engine, comprising several stages:

  • Layout analysis, tokenisation, lexical analysis: Standard text processing steps to break down documents into manageable units.
  • Name recognition and classification: Identifying and categorizing entities like diseases, locations, and affected populations.
  • Shallow syntactic analysis: Understanding the grammatical structure of sentences.
  • Resolution of co-reference among entities: Linking different mentions of the same entity within a text.
  • Pattern-based event matching and role mapping: Identifying specific outbreak events and assigning roles to the recognized entities (e.g., "disease X occurred in location Y").
  • Normalisation and output generation: Standardizing the extracted facts and formatting them for the database.

What's novel is the integration of machine learning at several stages to enhance performance and data quality. This includes:

  • Domain knowledge acquisition: Using machine learning to populate knowledge bases.
  • Automatic post-validation: Employing machine learning to detect and reduce errors in the extracted facts.
  • Confidence notion: Assigning a confidence score to individual extracted facts.
  • Aggregation of facts: Combining separate facts from multiple reports into a single, comprehensive outbreak record, leveraging confidence scores.

Datasets

The system utilizes reports from ProMED-Mail, a mailing list about epidemic outbreaks maintained by the International Society for Infectious Diseases. This is described as one of the most comprehensive sources of such reports, with over 10 years of data. The paper does not explicitly mention separate datasets for training or evaluation, but implies that the large volume of ProMED-Mail data is used for both the unsupervised learning components and for evaluating the system's performance in a large-scale setting.

Key Insights and Ideas

  • Leveraging redundancy for accuracy: The system demonstrates that even noisy extracted data can be improved by analyzing global trends and correcting local mis-analyses. This highlights the power of aggregating information from many sources.
  • The necessity of large-scale processing: The authors emphasize that operating "in the large" is crucial for the effectiveness of their machine learning components, which rely on abundant data for training.
  • Bridging IE and KDD: The work advocates for tighter integration between Information Extraction and Knowledge Discovery in Databases (KDD) components, suggesting mutual benefits for both fields.
  • Publicly accessible epidemiological database: The creation of a large-scale, on-line, publicly accessible IE database for infectious disease outbreaks is a significant contribution, enabling broader research.

Key Results

The paper describes the ProMED-PLUS system and its architecture, highlighting its ability to integrate off-line and on-line components to serve as a base for research. The system extracts facts from ProMED-Mail reports and makes the database available on-line via a web server. Users can view, select, sort, and plot outbreaks and individual incidents. All extracted facts are linked back to their original reports. The system's ability to aggregate facts across multiple reports based on confidence is a key feature for building comprehensive outbreak records.

Limitations and Future Work

The paper acknowledges that automatic error reduction requires a "critical mass of extracted facts." While not explicitly stated as a limitation, the reliance on machine learning for various stages implies that the system's performance is dependent on the quality and quantity of available training data. Future work could involve further refining the machine learning models, expanding the range of extractable information, and potentially integrating with other data sources to provide an even more comprehensive view of infectious disease outbreaks.

Summary updated

BibTeX
@inproceedings{yangarber2005extracting,
  author = {Roman Yangarber and Lauri Jokipii and Antti Rauramo and Silja Huttunen},
  title = {{Extracting Information about Outbreaks of Infectious Epidemics}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}
Figure