TL;DR
This paper presents a system for automatically extracting researcher affiliation information from academic conference Calls-for-Papers (CFPs). By combining heuristic approaches with named entity recognition and pattern discovery, the system aims to build a more comprehensive and accurate research data management system.
What Problem Does This Paper Address?
Maintaining accurate and up-to-date information about researchers' affiliations is a significant challenge for research data management systems (RDMSs). Traditionally, this relies on manual updates, which are time-consuming and prone to errors. The authors identify Calls-for-Papers (CFPs) as a rich, readily available, and frequently updated source of this information. However, CFPs are often semi-structured natural language documents, making automated extraction difficult. This paper tackles the problem of extracting researcher and organization affiliation data from these documents to populate and maintain RDMSs.
Methods
The proposed system employs a multi-phase approach, integrating information extraction (IE) and entity resolution (ER).
The Information Extraction phase consists of:
- Named Entity Recognition (NER): Utilizes the Stanford Named Entity Recognizer to identify tokens as People (P), Organizations (O), or Locations (L). It also identifies layout features like newlines and commas.
- Region Detection: Identifies "high-density regions" (HDRs) within the document that likely contain affiliation data by looking for sequences of tokens with a significant proportion of entities.
- Pattern Discovery: Analyzes the detected HDRs to find the most common patterns (e.g., "P, O" or "P (O,L)") used in listing program committee members. This helps in understanding the structure of the data.
- NER Correction: Uses the discovered patterns to correct misclassified entities identified by the NER module. It also uses delimiters and record boundary detection to group tokens into fields.
- Database Population: Extracts the identified entities and relationships and stores them in a relational database. This involves matching extracted entities against existing entries in the database using various string matching and similarity strategies.
The Entity Resolution phase, executed periodically, aims to resolve errors and inconsistencies in the database, such as duplicate entries for the same person or organization, missing relationships, or incorrect data. It uses string-similarity methods like Levenshtein distance and Jaccard similarity.
Datasets
The paper mentions using the following for seeding the database:
- GeoNames database: For populating the Locations table.
- Universities Worldwide database: For populating the Groups (organizations) table.
- DBLP database: For populating the People table.
For evaluation, the system was tested on 100 CFP emails, which contained 754 distinct affiliation records. The paper notes that these emails included duplicates and near-duplicates, which aided in evaluating the system's ability to recognize repeated records.
Key Insights and Ideas
- Exploiting CFP Structure: CFPs, while semi-structured, offer more predictable patterns than free-form text, which can be leveraged for more accurate information extraction.
- Multi-phase Entity Resolution: Separating entity resolution into an initial extraction phase and a later background consolidation phase allows for iterative refinement of the data.
- Combining Content and Layout: The system effectively combines named entity recognition (content) with layout features (structure, punctuation, line breaks) to improve extraction accuracy.
- Pattern-based Correction: Using discovered patterns to correct NER errors is a novel approach that can overcome limitations of standalone NER systems.
- Iterative Improvement: The system is designed to improve over time as more data is added to the database, enabling more robust entity resolution.
Key Results
The paper reports preliminary results on the information extraction phase using 100 CFP emails. The system achieved:
- Precision: 0.83
- Recall: 0.89
- F-measure: 0.89
These metrics indicate a good balance between correctly extracted records and missed records. The authors note that the system is effective in extracting affiliation information, even with the presence of duplicates and near-duplicates in the input data.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Multiple Patterns in a Region: If multiple valid patterns exist within a single region, the system's current approach of selecting only one pattern might lead to misinterpretation or ignoring of some records. Future work could involve assigning weights to patterns.
- Insufficient Entity Identification: The NER system might not identify enough entities, potentially hindering the formation of high-density regions. Incorporating more layout attributes could help.
- Structured Text: The current evaluation was on plain text CFPs. Future work should focus on adapting the system to handle structured text formats like HTML.
- Broader RDMS Integration: The system is part of a larger research data management system, and further work is needed to fully integrate and leverage the extracted affiliation data within that context.
Summary updated