Traditional Named Entity Recognition (NER) operates under a straightforward assumption: an entity's surface form or semantic category defines its identity within a text. If a model encounters "ethanol," it labels it a CHEMICAL. If it encounters "flour," it labels it an INGREDIENT.
However, in procedural, technical, and highly contextual domains, knowing what an entity is provides only half the picture. To truly understand the text, a system must discover what the entity is doing. As Bian et al. observe, there is a fundamental ontological separation between a substance's identity and its context-dependent function. A single chemical compound can act as an active ingredient in one formulation and an inactive excipient in another.
When models rely purely on surface forms, they suffer from functional role ambiguity. This article explores how modern information extraction models move beyond static entity typing to dynamically map entities to their functional roles based on context.
The Failure of Static Entity Recognition
In chemical synthesis patents, a compound like sodium hydroxide might serve as a starting material, a catalyst, or a solvent depending entirely on the specific step of the reaction sequence. As Nittala et al. and Li et al. point out, standard NER datasets that merely tag chemical names without specifying these functional roles leave models blind to the underlying logic of the procedure.
Static NER: [Sodium hydroxide]CHEMICAL was added to the solution...
Functional NER: [Sodium hydroxide]CATALYST was added to the solution...
Without resolving this ambiguity, it is impossible to reconstruct accurate, automated reaction graphs or workflows. If a system cannot distinguish between the substance being transformed (the reactant) and the substance facilitating the transformation (the catalyst), the resulting step-by-step model of the experiment will break down. This limitation is not unique to chemistry; it appears whenever text describes a process where entities interact dynamically.
How Contextual Attention Resolves Ambiguity
How do modern natural language processing systems capture these dynamic roles? The answer lies in how Transformer-based architectures leverage bidirectional contextual representations and self-attention mechanisms.
Consider two sentences describing a chemical process:
- "Compound A was dissolved in ethanol, and the mixture was heated."
- "Ethanol was oxidized to produce acetaldehyde."
In isolation, the token "ethanol" is identical in both sentences. However, the surrounding words—specifically operational verbs (dissolved in, oxidized) and prepositions (in, to)—contain the critical cues that dictate its function.
Work by Ruas et al. demonstrates that bidirectional context modeling allows language models to look both forward and backward in a sequence, weighing these surrounding syntactic and semantic cues simultaneously. Instead of mapping a token directly to a static class based on its character composition, the model evaluates the token in tandem with its environment.
This process is driven by self-attention. As highlighted by He et al. and He et al., attention mechanisms successfully capture long-range dependencies across a sentence or paragraph. When computing the vector representation for an entity, the attention mechanism assigns high weights to the specific verbs or procedural steps associated with it.
If "ethanol" attends strongly to "dissolved in," the architecture shifts its hidden state to reflect the role of a solvent. If it attends strongly to "oxidized," its state shifts toward a starting material. Dutta et al. confirm that this attention-based decoding is far more effective at capturing these relational dependencies than traditional sequence labeling methods like linear-chain Conditional Random Fields (CRFs), which often struggle with long-distance dependencies.
Mapping Entities to Functional Slots
Once a model can resolve these contextual nuances, the task shifts from simple sequence tagging to slot filling and relationship labeling. The goal is to map entities directly into functional slots that define their operational purpose within a broader system.
In materials science, for example, Friedrich et al. show that identifying a material is insufficient unless it can be mapped to specific experimental slots, such as determining whether a compound functions as an anode or a cathode. Similarly, in the culinary domain, Fan et al. utilize contextualized sequence classification to determine whether an ingredient or object functions as a tool, a target, or a destination within a recipe workflow.
"Pour the mixture into the [pan]DESTINATION" vs. "Heat the [pan]TOOL"
This slot-filling approach transforms flat text into structured, executable knowledge graphs. However, achieving this requires moving beyond surface-level syntax. As Zhong et al. note, sentence structure alone can sometimes be deceptive, requiring models to integrate weak supervision or domain-specific knowledge to accurately differentiate complex roles like catalysts from reactants when the syntactic framing is identical.
A Ubiquitous Pattern Across Domains
While heavily pronounced in scientific literature, the challenge of functional role ambiguity spans across diverse text types wherever intent and context override inherent noun definitions.
- Voicemail Information Extraction: Huang et al. found that standard NER fails to isolate the primary actor or callback target from a list of mentioned names in a messy transcript without modeling the underlying intent of the speaker.
- Cybercrime Marketplaces: Durrett et al. demonstrated that common nouns shift between benign terms and specific illicit products based entirely on transactional verbs surrounding them in specialized forums.
- Document Structure: At a macro level, Cohan et al. showed that the functional role of an entire sentence (e.g., whether it serves as a method, a result, or background background) cannot be determined in isolation, but must be resolved by attending to the context of neighboring sentences.
Ultimately, context-dependent functional role labeling works because it treats language not as a collection of static, labeled objects, but as a dynamic sequence of operations where an entity's meaning is defined entirely by its utility in context.
Key Foundational Work
Information Extraction from Voicemail (2001) — contributed 1 concept elements to this direction.
An accurate and precise representation of drug ingredients (2016) — contributed 1 concept elements to this direction.