TL;DR
This paper introduces PROFILE, a system that automatically builds a knowledge source of entity descriptions from internet-accessible newswire. By extracting and storing these descriptions, PROFILE aims to improve natural language generation and summarization systems by providing richer, contextually relevant information about entities.
What Problem Does This Paper Address?
Current natural language generation (NLG) and summarization systems often struggle when they need to refer to entities (people, places, organizations) that might not have been fully described in the input text. Information that is crucial for a reader's understanding, especially in news summaries, might be missing or insufficient. While external online sources could provide this missing information, integrating it seamlessly into a generation system is challenging. This paper addresses the need for an automated way to gather and structure this external information, creating a "knowledge source" that NLG systems can readily access and utilize.
Methods
The core of the PROFILE system involves several key steps:
- Entity Extraction: The system first identifies potential entities (proper nouns) within a corpus of newswire articles. It uses part-of-speech tagging and regular grammars (CREP) to extract noun phrases. To filter out non-entity candidates (like titles), it leverages WordNet for semantic filtering.
- Description Extraction: Once entities are identified, the system extracts descriptive phrases associated with them. This is done using finite-state techniques to capture noun phrases that act as pre-modifiers or appositives to the entity. This extraction can happen either from a pre-existing corpus or in real-time from live newswire feeds.
- Description Categorization: To make descriptions more useful for generation, they are semantically categorized. The system uses WordNet to find hypernyms for words within a description, identifying concepts like "profession," "nationality," or "organization."
- Profile Database Construction: Extracted and categorized descriptions are stored in a database of "profiles," where each profile is keyed by an entity name. The database stores the surface string of the description, its source, date, and frequency.
- Functional Description (FD) Generation: A crucial step for NLG is converting the extracted descriptions into a structured format. PROFILE converts these descriptions into functional descriptions (FDs), a representation suitable for input to generation systems like FUF/SURGE. This allows for more flexible manipulation and integration of descriptions compared to using raw text snippets.
Datasets
- Reuters Newswire Corpus: A 1.4 MB corpus of Reuters newswire from March to June 1995 was used to seed the initial database of entity profiles.
- Live Newswire Sources: The system is designed to access live newswire feeds from sources like Reuters (via Yahoo.com), CNN Interactive (www.cnn.com), and ClariNet (via NNTP) for real-time information extraction.
Key Insights and Ideas
- Bridging the Gap in Generation: The most significant insight is that by automatically building a knowledge source of entity descriptions, PROFILE can significantly enhance the quality and completeness of generated text, especially in domains like news summarization where context is key.
- Real-time Information Integration: The system's ability to connect to live newswire feeds allows it to incorporate the latest information, making it highly relevant for dynamic content like breaking news.
- Structured Representations for Flexibility: Converting extracted descriptions into Functional Descriptions (FDs) is a key innovation. This structured format enables grammatical transformations (like aggregation) and semantic unification, allowing for more sophisticated generation than simply inserting canned phrases.
- Semantic Categorization for Relevance: Using WordNet to categorize descriptions helps the generation system select the most relevant description based on semantic features, improving the coherence and appropriateness of the generated text.
Key Results
- Entity Extraction Precision: The paper reports that after semantic filtering using WordNet, the system achieved a list of 421 unique entity names from the initial corpus, all of which were confirmed as proper nouns.
- Description Extraction Precision: A random evaluation showed that 551 out of 611 (90.2%) retrieved descriptions were correct.
- FD Generation: The system successfully converts extracted descriptions into functional descriptions, demonstrated with an example for "Silvio Berlusconi." These FDs can be used by the FUF/SURGE generation system.
Limitations and Future Work
- Entity Cross-referencing: The current system does not handle entity cross-referencing, meaning it won't automatically link "Clinton" and "Bill Clinton" or connect a person's profile to an organization they belong to.
- Polysomy: The system struggles with polysemy, potentially miscategorizing phrases like "Rice University" because the word "rice" can also refer to food.
- FDs Not Yet Used in Summarization: While the system generates FDs, they are not yet integrated into a summarization system.
- Future Work: The authors plan to:
- Increase coverage by adding more online news sources and developing a comprehensive, shareable profile database accessible via the Web.
- Refine algorithms to handle polysemy and improve entity cross-referencing.
- Investigate algorithms for selecting and ordering descriptions for evolving summaries and for generating referring expressions.
- Implement mechanisms to match different instances of the same entity (e.g., "PLO" and "Palestine Liberation Organization").
- Explore using other protocols like SMTP for description retrieval and integrate with search engines like Lycos and AltaVista.
- Develop KQML interfaces for better integration with other information access modules.
Summary updated