Abstract
Experimental research publications provide figure form resources including graphs, charts, and any type of images to effectively support and convey methods and results. To describe figures, authors add captions, which are often incomplete, and more descriptions reside in body text. This work presents a method to extract figure descriptive text from the body of scientific articles. We adopted ontological semantics to aid concept recognition of figure-related information, which generates human- and machine-readable knowledge representations from sentences. Our results show that conceptual models bring an improvement in figure descriptive sentence classification over word-based approaches.
Introduction
In scientific literature, figures such as graphs and charts are primary vehicles for conveying complex experimental results. However, a persistent "information gap" exists: figure captions are frequently concise to the point of being incomplete. To fully comprehend a figure, a reader must often locate and synthesize explanations buried within the surrounding body text—sentences that clarify axis representations, trend significances, or specific symbol meanings.
Automating the extraction of this descriptive text is challenging because these sentences do not always contain explicit keywords like "Figure." For instance, a sentence stating, "The spectra show a prominent resonant microwave absorption signal..." clearly describes a plot but would be missed by simple keyword-based searches. This research aims to bridge this gap by moving beyond "bag-of-words" approaches toward a system that understands the underlying concepts of description.
Methodology
The proposed method utilizes Ontological Semantic Technology (OST) to analyze the semantic structure of sentences. The extraction pipeline consists of several key stages:
- Sentence Selection: The system identifies "figure-referring sentences" (e.g., "See Fig. 1") and evaluates a window of two sentences before and after as candidates for descriptive text.
- Text Meaning Representation (TMR): The system generates TMRs—graph-based representations of a sentence's logic—identifying subjects, verbs, objects, and modifiers.
- Ontological Mapping: These components are mapped to a language-independent ontology. For example, the lexeme "straight" maps to the property
SHAPE. - Handling Lexical Gaps: If a verb is missing from the lexicon, the system employs WordNet and Word2Vec to infer the concept. Placeholder "UNKNOWN" entities are used to maintain path connectivity in the knowledge graph when logic gaps occur.
- Scoring: Concepts (e.g.,
DRAWING) and properties (e.g.,DIRECTIONALITY) are assigned weights. A total "Sentence Weight" is calculated; if it exceeds a threshold $\lambda$, the sentence is classified as descriptive.
Experimental Setup and Results
The study focused on a corpus of 40,561 papers in the domain of X-ray absorption spectroscopy (XAS). Evaluation was performed on manually labeled sets of 200 sentences each from both XML and PDF formats to test robustness across different data structures.
The conceptual model consistently outperformed traditional "bag-of-words" machine learning approaches. On the XML dataset, the model achieved an F-score of 0.82 (at $\lambda = 0.5$). Even on the more challenging PDF format, which lacks structural tags, it maintained a respectable F-score of 0.78.
In a direct comparison with standard machine learning algorithms like Support Vector Machines (SVM) and Random Forests (RF), the conceptual model (0.797 F-score) significantly outperformed the best-performing traditional model, an SVM (0.679 F-score).
Discussion and Conclusion
The core finding is that descriptive text possesses a distinct "conceptual signature." Sentences describing figures frequently utilize concepts related to shape, rate of change, and graphical symbols. By weighting these ontological properties, the system identifies descriptions even without explicit figure references.
While highly effective, the current model is limited to single-sentence analysis and does not yet account for descriptions spanning multiple sentences. Future work will aim to expand the ontology to other scientific domains and move beyond identifying descriptive text toward extracting interpretive text that explains the broader implications of the data presented in figures.
Summary updated