TL;DR
This paper reveals significant structural and content differences between biomedical journal article abstracts and full-text bodies. These differences, particularly longer sentences and extensive parenthetical content in full texts, pose challenges but also offer opportunities for text mining, impacting the performance of tools like gene mention taggers and POS taggers.
What Problem Does This Paper Address?
The field of biomedical natural language processing (NLP) has historically focused on abstracts due to their accessibility and conciseness. However, with the growth of full-text repositories like PubMedCentral, there's a growing opportunity to leverage the richer information contained within full articles. Before this study, a comprehensive characterization of how full-text article bodies differ from abstracts in terms of structure, content, and their implications for text mining tools was lacking. This paper aims to fill that gap by systematically comparing abstracts and full-text bodies across various linguistic and semantic features, and evaluating the performance of NLP tools on each.
Methods
The researchers employed a multi-faceted approach to compare abstracts and full-text article bodies:
- Structural Analysis: They analyzed sentence lengths and the incidence and types of parenthesized text. Sentence segmentation was performed using LingPipe, and sentence lengths were measured in words. Parenthesized text was categorized using regular expressions into nine types, with manual validation for accuracy.
- Morphosyntactic and Discourse Feature Analysis: They examined the incidence of coordination (using "and," "or," "but"), passives (identified by "ed by"), negation (using "no," "not," "neither"), and pronominal anaphora (using a simple pronoun search). Statistical tests (t-tests and Mann-Whitney-Wilcox tests) were used to determine significant differences.
- Semantic Class Distribution: They analyzed the distribution of four semantic classes: gene/protein names, mutations, drug names, and diseases. This involved using existing tools like ABNER, BANNER, and MutationFinder, and a dictionary-based approach for drug names.
- Text Mining Tool Performance Evaluation: They assessed the performance of gene mention taggers (ABNER, BANNER, LingPipe) and a mutation detection system (MutationFinder) on both abstracts and full-text bodies. They also evaluated the performance of a part-of-speech (POS) tagger.
Datasets
The primary dataset used was the CRAFT corpus, consisting of 97 full-text biomedical journal articles. This corpus was specifically selected for its relevance to mouse genomics and curated by the Mouse Genome Informatics (MGI) database. The articles were available in PubMedCentral XML format, allowing for segmentation into abstract and body sections. For drug mentions, the DrugCards data from DrugBank was used as a dictionary.
Key Insights and Ideas
- Full text is structurally different: Article bodies feature significantly longer sentences and a much higher incidence of parenthesized text compared to abstracts. This suggests that standard NLP tools trained on abstracts might struggle with the complexity of full texts.
- Parentheses are a goldmine (and a challenge): While parenthesized text can be a challenge for parsers, it also contains valuable information (e.g., gene symbols, data values, citations) that is largely absent in abstracts. This presents an opportunity for targeted information extraction.
- Linguistic features vary: The frequency of passives and negation is significantly higher in article bodies, which are known to be challenging for NLP systems.
- Semantic content is unevenly distributed: While genes are found in both abstracts and bodies, mutations, drugs, and diseases are far more prevalent in the full text. This highlights the need to process full text for comprehensive biomedical knowledge extraction.
- Tool performance is genre-dependent: Gene mention taggers and POS taggers generally perform worse on full-text articles than on abstracts, indicating a need for genre-specific training or adaptation.
Key Results
- Sentence Length: Article bodies have significantly longer sentences than abstracts. For example, median sentence lengths in the abstract were shorter than in most body sections.
- Parenthesized Text: Parenthesis usage in article bodies was nearly triple that in abstracts (28 per thousand words vs. 10 per thousand words). Abbreviations and gene symbols were the most common type in abstracts (55%), while figure/table pointers dominated in bodies (over 50%).
- Linguistic Features:
- Passives: Higher incidence in bodies (4.3/thousand words) than abstracts (3.7/thousand words).
- Negation: Higher incidence in bodies (5.3/thousand words) than abstracts (3.8/thousand words).
- Pronominal Anaphora: Higher incidence in abstracts (5.3/thousand words) than bodies (3.98/thousand words).
- Coordination: No statistically significant difference in incidence.
- Semantic Classes:
- Mutations: Only 1 abstract mentioned mutations, while 19 article bodies did.
- Drugs: 19/97 abstracts mentioned drugs, compared to 85/97 bodies.
- Diseases: Significantly more mentions in bodies (average 23 per body) than abstracts (average 1 per abstract).
- Tool Performance:
- Gene Mention Taggers: F-measures were generally 10 points higher on abstracts than on bodies.
- Mutation Finder: Performed similarly well on both genres (precision of 1.00 on abstracts and 0.959 on full text).
- POS Tagging: Mean tag accuracy was 83% on abstracts and 81% on bodies, with the difference being statistically significant.
Limitations and Future Work
The authors acknowledge that this study is primarily descriptive and does not directly evaluate the impact of these differences on specific NLP tasks like information extraction or retrieval. They suggest that future work should focus on developing better parsers, improving the handling of passives and negation, developing methods to effectively process parenthesized text, and retraining gene mention systems and other taggers on full-text data. The creation of more comprehensive full-text corpora is also highlighted as crucial.
Summary updated