The structural and content aspects of abstracts versus bodies of full text journal articles are different

K. Bretonnel Cohen, Helen L. Johnson, Karin Verspoor, Christophe Roeder, Lawrence Hunter

TL;DR

This paper reveals significant structural and content differences between biomedical journal article abstracts and full-text bodies. These differences, particularly longer sentences and extensive parenthetical content in full texts, pose challenges but also offer opportunities for text mining, impacting the performance of tools like gene mention taggers and POS taggers.

What Problem Does This Paper Address?

The field of biomedical natural language processing (NLP) has historically focused on abstracts due to their accessibility and conciseness. However, with the growth of full-text repositories like PubMedCentral, there's a growing opportunity to leverage the richer information contained within full articles. Before this study, a comprehensive characterization of how full-text article bodies differ from abstracts in terms of structure, content, and their implications for text mining tools was lacking. This paper aims to fill that gap by systematically comparing abstracts and full-text bodies across various linguistic and semantic features, and evaluating the performance of NLP tools on each.

Methods

The researchers employed a multi-faceted approach to compare abstracts and full-text article bodies:

  • Structural Analysis: They analyzed sentence lengths and the incidence and types of parenthesized text. Sentence segmentation was performed using LingPipe, and sentence lengths were measured in words. Parenthesized text was categorized using regular expressions into nine types, with manual validation for accuracy.
  • Morphosyntactic and Discourse Feature Analysis: They examined the incidence of coordination (using "and," "or," "but"), passives (identified by "ed by"), negation (using "no," "not," "neither"), and pronominal anaphora (using a simple pronoun search). Statistical tests (t-tests and Mann-Whitney-Wilcox tests) were used to determine significant differences.
  • Semantic Class Distribution: They analyzed the distribution of four semantic classes: gene/protein names, mutations, drug names, and diseases. This involved using existing tools like ABNER, BANNER, and MutationFinder, and a dictionary-based approach for drug names.
  • Text Mining Tool Performance Evaluation: They assessed the performance of gene mention taggers (ABNER, BANNER, LingPipe) and a mutation detection system (MutationFinder) on both abstracts and full-text bodies. They also evaluated the performance of a part-of-speech (POS) tagger.

Datasets

The primary dataset used was the CRAFT corpus, consisting of 97 full-text biomedical journal articles. This corpus was specifically selected for its relevance to mouse genomics and curated by the Mouse Genome Informatics (MGI) database. The articles were available in PubMedCentral XML format, allowing for segmentation into abstract and body sections. For drug mentions, the DrugCards data from DrugBank was used as a dictionary.

Key Insights and Ideas

  • Full text is structurally different: Article bodies feature significantly longer sentences and a much higher incidence of parenthesized text compared to abstracts. This suggests that standard NLP tools trained on abstracts might struggle with the complexity of full texts.
  • Parentheses are a goldmine (and a challenge): While parenthesized text can be a challenge for parsers, it also contains valuable information (e.g., gene symbols, data values, citations) that is largely absent in abstracts. This presents an opportunity for targeted information extraction.
  • Linguistic features vary: The frequency of passives and negation is significantly higher in article bodies, which are known to be challenging for NLP systems.
  • Semantic content is unevenly distributed: While genes are found in both abstracts and bodies, mutations, drugs, and diseases are far more prevalent in the full text. This highlights the need to process full text for comprehensive biomedical knowledge extraction.
  • Tool performance is genre-dependent: Gene mention taggers and POS taggers generally perform worse on full-text articles than on abstracts, indicating a need for genre-specific training or adaptation.

Key Results

  • Sentence Length: Article bodies have significantly longer sentences than abstracts. For example, median sentence lengths in the abstract were shorter than in most body sections.
  • Parenthesized Text: Parenthesis usage in article bodies was nearly triple that in abstracts (28 per thousand words vs. 10 per thousand words). Abbreviations and gene symbols were the most common type in abstracts (55%), while figure/table pointers dominated in bodies (over 50%).
  • Linguistic Features:
    • Passives: Higher incidence in bodies (4.3/thousand words) than abstracts (3.7/thousand words).
  • Negation: Higher incidence in bodies (5.3/thousand words) than abstracts (3.8/thousand words).
  • Pronominal Anaphora: Higher incidence in abstracts (5.3/thousand words) than bodies (3.98/thousand words).
  • Coordination: No statistically significant difference in incidence.
  • Semantic Classes:
    • Mutations: Only 1 abstract mentioned mutations, while 19 article bodies did.
  • Drugs: 19/97 abstracts mentioned drugs, compared to 85/97 bodies.
  • Diseases: Significantly more mentions in bodies (average 23 per body) than abstracts (average 1 per abstract).
  • Tool Performance:
    • Gene Mention Taggers: F-measures were generally 10 points higher on abstracts than on bodies.
  • Mutation Finder: Performed similarly well on both genres (precision of 1.00 on abstracts and 0.959 on full text).
  • POS Tagging: Mean tag accuracy was 83% on abstracts and 81% on bodies, with the difference being statistically significant.

Limitations and Future Work

The authors acknowledge that this study is primarily descriptive and does not directly evaluate the impact of these differences on specific NLP tasks like information extraction or retrieval. They suggest that future work should focus on developing better parsers, improving the handling of passives and negation, developing methods to effectively process parenthesized text, and retraining gene mention systems and other taggers on full-text data. The creation of more comprehensive full-text corpora is also highlighted as crucial.

Summary updated

Abstract

Background An increase in work on the full text of journal articles and the growth of PubMedCentral have the opportunity to create a major paradigm shift in how biomedical text mining is done. However, until now there has been no comprehensive characterization of how the bodies of full text journal articles differ from the abstracts that until now have been the subject of most biomedical text mining research. Results We examined the structural and linguistic aspects of abstracts and bodies of full text articles, the performance of text mining tools on both, and the distribution of a variety of semantic classes of named entities between them. We found marked structural differences, with longer sentences in the article bodies and much heavier use of parenthesized material in the bodies than in the abstracts. We found content differences with respect to linguistic features. Three out of four of the linguistic features that we examined were statistically significantly differently distributed between the two genres. We also found content differences with respect to the distribution of semantic features. There were significantly different densities per thousand words for three out of four semantic classes, and clear differences in the extent to which they appeared in the two genres. With respect to the performance of text mining tools, we found that a mutation finder performed equally well in both genres, but that a wide variety of gene mention systems performed much worse on article bodies than they did on abstracts. POS tagging was also more accurate in abstracts than in article bodies. Conclusions Aspects of structure and content differ markedly between article abstracts and article bodies. A number of these differences may pose problems as the text mining field moves more into the area of processing full-text articles. However, these differences also present a number of opportunities for the extraction of data types, particularly that found in parenthesized text, that is present in article bodies but not in article abstracts.
BibTeX
@article{cohen2010structural,
  author = {K. Bretonnel Cohen and Helen L. Johnson and Karin Verspoor and Christophe Roeder and Lawrence Hunter},
  title = {{The structural and content aspects of abstracts versus bodies of full text journal articles are different}},
  journal = {BMC Bioinformatics},
  year = {2010},
  url = {https://openreview.net/forum?id=ry7PYFznBnz},
}
Figure