Abstract: Daivik Swarup, Ahsaas Bajaj, Sheshera Mysore, Tim O’Gorman, Rajarshi Das, Andrew McCallum. Findings of the Association for Computational Linguistics: EMNLP 2020.
2020.
Introduction
In experimental biology and materials science, researchers document their work in procedural texts (e.g., "Centrifuge the sample at 14,000xg for 5 minutes"). Automating the extraction of structured information from these texts—a task known as shallow semantic parsing—is critical for mining scientific literature and automating laboratory protocols. However, two major hurdles persist: extreme data scarcity in niche domains and the complexity of generalizing from limited examples of specific scientific relations.
The "Sublanguage" Hypothesis
Scientific protocols are often written in a "sublanguage"—a highly repetitive, semi-structured, and formulaic style. The authors observe that if two sentences are structurally similar, the relationships between the entities within them are likely identical. This insight suggests that a model capable of leveraging local, instance-level similarities could outperform standard global parametric models.
Methodology: A Hybrid "Copy-Paste" Architecture
The proposed system addresses relation prediction (identifying links between entities) through a hybrid architecture:
- Instance-Level (Local) Model: Using SciBERT embeddings, the system identifies the most similar sentences in the training set (nearest neighbors). If a neighbor contains a labeled relation, the model learns to "copy" that label to the test instance.
- Parametric (Global) Model: A standard feed-forward network provides a "best guess" based on learned global patterns, serving as a fallback when neighbors do not provide a clear label.
- Hybrid Logic: Unlike previous models that use complex gating mechanisms, this approach is explicitly biased toward the local neighborhood. It prioritizes the local score if a label exists in the neighborhood, falling back to the global model only when necessary.
Experimental Setup and Results
The model was evaluated on two specialized corpora: the Wet Lab Protocol (WLP) corpus (622 protocols) and the Materials Science Procedural Text (MSPT) corpus (230 procedures).
Performance Comparison
As shown in the table below, the hybrid method achieved state-of-the-art results on both datasets, outperforming previous benchmarks and standard global-only models.
Data Efficiency
A key strength of the instance-level approach is its robustness in low-data regimes. The "Our Method" (OM) approach consistently outperformed the global model (GM) even when training data was reduced to as little as 5% of the original set.
Key Insights
- Formulaic Writing as a Feature: In scientific protocols, the constrained nature of language makes "copying" a powerful tool for precision.
- Inductive Bias: Forcing the model to prioritize the local neighborhood provides a strong human-designed bias that outperforms flexible but data-hungry gate mechanisms in low-data scenarios.
- Precision over Recall: The instance-based approach is naturally conservative, leading to higher precision by relying on near-identical historical examples.
Conclusion
This hybrid approach demonstrates that leveraging the formulaic nature of scientific sublanguages can significantly mitigate the challenges of data scarcity in semantic parsing. Future work may explore dynamic retrieval systems and extending the model to predict higher-order structures like complex event chains.
Summary updated