TL;DR
This paper investigates whether scientific abstracts capture all the important information in a research paper, or if "peer-reviewed" insights from citations (citances) reveal more. They found that citances collectively contain about 20% more biological concepts than abstracts, suggesting that peers indeed see more in a paper than its authors initially present. This has implications for how we summarize and understand scientific literature.
What Problem Does This Paper Address?
As more scientific literature becomes freely accessible (open access), researchers are increasingly interested in automatically extracting valuable information from these texts. A key question is how informative a paper's abstract is compared to its full text. Specifically, the authors want to know:
- What crucial information from the full text is missing from the abstract?
- What should an ideal summary of a paper include, beyond its abstract?
- Do authors (who write the abstract) and their peers (who cite the paper) perceive the paper's contributions differently?
They tackle this by analyzing "citances" – the sentences in other papers that cite a target article – as a proxy for the "peer viewpoint."
Methods
The researchers employed a two-pronged approach:
- Manual Analysis: They meticulously annotated abstracts and their corresponding citances for six biomedical articles in the molecular interaction domain. This involved identifying and categorizing specific biological concepts like entities (genes, proteins), functions, experimental methods, chemicals, and disorders. They developed a detailed annotation scheme and rules to ensure consistency.
- Automatic Analysis: They scaled up their analysis to 104 biomedical articles. Here, they used MeSH (Medical Subject Headings), a standardized hierarchical vocabulary, to automatically identify and map concepts in abstracts and citances. This allowed for a broader, quantitative comparison.
Both methods focused on comparing the content of abstracts to the content of citances, rather than just citation counts. They also investigated the influence of "adjoining citations" (citations to other papers within the same citance sentence) and the passage of time on the information conveyed in citances.
Datasets
- Manual Analysis Dataset: 6 biomedical journal abstracts and 136 associated citances from the molecular interaction domain, published between 1996-2002.
- Automatic Analysis Dataset: 104 biomedical journal articles from the molecular interaction domain, published between 1995-2002, with a total of 11,199 citances collected from 1995-2005.
The authors leveraged existing MeSH terms for automatic annotation and compared their manual categories to MeSH for mapping.
Key Insights and Ideas
- Citances as a "Peer Viewpoint": The core idea is that sentences citing a paper (citances) reflect what other researchers found important enough to highlight, offering a valuable, external perspective distinct from the author's self-summary in the abstract.
- Abstracts are Author-Biased: Abstracts, written by authors, tend to be broader but may focus on a smaller set of concepts compared to the collective insights from peers.
- Full Text Holds More Than Abstracts: The significant overlap between abstracts and citances suggests abstracts capture much of the core message. However, citances consistently contain additional information not present in the abstract, implying this information originates from the full text and is deemed important by peers.
- "Adjoining Citations" Add More Information: Citances that also cite other papers tend to introduce even more diverse concepts, suggesting a richer context when multiple references are discussed together.
- Time Evolves Citances: While initial citations might align closely with abstracts, citances over time can diverge, potentially reflecting a more objective or evolving understanding of a paper's impact.
Key Results
- 20% More Concepts in Citances: The manual analysis revealed that citances collectively contained approximately 20% more biological concepts than the abstracts of the same papers.
- Entities and Methods Dominate Extra Information: The additional information found in citances, not present in abstracts, was primarily related to "entities" (like specific genes or proteins) and "experimental methods."
- High Overlap: Despite the extra information in citances, there was a substantial overlap between abstract and citance annotations, with citances covering at least 50% of the abstract's unique annotations, and often around 75-80%.
- Automatic Analysis Confirms Manual Findings: The large-scale automatic analysis using MeSH terms corroborated the manual findings, confirming that citances provide more comprehensive information, particularly regarding biological entities and experimental methods.
Limitations and Future Work
- Assumption of Full Text Reliance: The study assumes that peers cite based on the full text, not just the abstract. While a strong assumption, they believe it generally holds.
- Citances as Atomic Units: The analysis treated citance sentences as atomic units, even if they cited multiple papers. Future work could explore identifying which part of a citance is relevant to a specific citation.
- Focus on Molecular Interactions: The study was limited to the molecular interaction domain. Applying the methodology to other scientific disciplines is a key future direction.
- Exploring Citation Repetitiveness: The authors plan to investigate the repetitiveness of citations to assess objectivity and importance more rigorously.
- Finer-Grained Analysis: Future work could involve more detailed MeSH categories and exploring resources like UMLS and GO for richer semantic annotations.
- Analyzing Predicate-Argument Relations: Moving beyond concepts to analyze relationships between them is another avenue for future research.
Summary updated