Historically, extracting structured information from text meant training specialized discriminative classifiers to label individual tokens (such as BIO tagging for Named Entity Recognition) or classify pairs of text spans (for Relation Extraction). This approach treats text as a sequence of discrete slots to be filled.
Generative Information Extraction (IE) flips this paradigm entirely. Instead of asking a model to classify tokens, it asks the model to translate raw text into a structured language. As surveyed by Derong Xu et al. in Large language models for generative information extraction: a survey, this framework unifies disparate extraction tasks—NER, relation extraction, event extraction, and sentiment analysis—into a single sequence-to-sequence (seq2seq) objective.
Why make this shift? Traditional token-level classification breaks down when data schemas become complex or overlapping. For instance, if a single entity belongs to multiple relations, standard sequence labeling struggles to assign multiple distinct tags to the same token. By treating extraction as a conditional generation task, as highlighted by Jianxin Li et al. in A Survey on Deep Learning Event Extraction: Approaches and Applications, the model can generate the same entity multiple times in different contexts within the output sequence, naturally bypassing the constraints of a fixed token label space.
The Art of Linearization
To make a seq2seq model output structured data (like graphs, tables, or triplets), the target structure must be flattened into a linear string of text. The choice of this "target language" determines how well the model learns the underlying extraction task.
Several linearization strategies have emerged across the literature:
- Inline Markup: Inserting special tokens directly into a copy of the input text. For example, Jinzhong Ning et al. in Breaking the Boundaries: A Unified Framework for Chinese Named Entity Recognition Across Text and Speech use special markers around recognized spans to capture nested entity structures directly within the text flow.
- Serialized Triplets: Generating a clean list of relational facts independent of the input sentence structure. In REBEL: Relation Extraction By End-to-end Language generation, Pere-Lluís Huguet Cabot et al. demonstrate that translating raw text into a linearized string of
subject-relation-objecttriplets allows an encoder-decoder transformer to perform end-to-end relation extraction. Similarly, Peter J Liu et al. in Assessing The Factual Accuracy of Generated Text train transformers to map paragraphs directly into serialized fact tuples. - Structured Formats (JSON and Tables): For downstream applications, models can be trained to output formal data formats. Liana Mikaelyan in Enhancing Enterprise Knowledge Base Construction with Fine-Tuned Generative Language Models shows that fine-tuning generative models to output structured JSON objects replaces rigid rule-based template matching with a flexible sequence mapping. For multi-table operations, Andrew Yates et al. in MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering flatten entire tables (headers and rows) into the output sequence to capture complex relational operations like joins.
Crucially, the authors of Towards Generative Aspect-Based Sentiment Analysis (Wenxuan Zhang et al.) note that generating structured lists of tuples is often more effective than reconstructing the entire input sentence with embedded labels. Avoiding sentence reconstruction reduces unnecessary decoder overhead, allowing the model to focus purely on recovering multi-attribute relationships.
Eliminating Pipeline Error Propagation
Traditional information extraction relies on a strict pipeline: first find the entities, then classify the relationships between them. If the entity extractor misses a word, the relation classifier never receives it. Errors cascade down the line.
Generative extraction collapses these steps into a single autoregressive generation pass. As Yilmazcan Ozyurt et al. argue in Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models, framing relation extraction as a triplet generation task completely bypasses the need for an explicit, isolated named entity recognition step.
By generating the entire relational structure at once, the model leverages the shared context of both the entity and the relation. Tapas Nayak et al. in Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction show that this unified sequence representation allows standard encoder-decoder architectures to extract multiple overlapping relationships simultaneously. Furthermore, Fanfan Wang et al. in Generative Emotion Cause Triplet Extraction in Conversations with Commonsense Knowledge emphasize that this single-pass sequence generation captures long-range dependencies across complex interactions, preventing the compounding errors inherent in multi-stage pipelines.
Handling Complexity, Nesting, and Normalization
Real-world text is rarely flat. Entities can be nested inside other entities (e.g., [University of [Texas]]), split across a sentence, or expressed using synonyms that do not match a rigid database schema.
Generative models excel here because the decoder is not constrained to selecting exact, contiguous spans from the source text. In a review of complex entity mentions, Xiang Dai 0001 in Recognizing Complex Entity Mentions: A Review and Future Directions notes that linearizing entity spans into a target sequence enables models to produce overlapping or nested mentions without specialized tagging schemas. This is further validated by Hang Yan et al. in A Unified Generative Framework for Various NER Subtasks, who use a pointer-based sequence generation framework to handle flat, nested, and discontinuous NER uniformly.
Beyond span extraction, generative models can perform normalization and entity resolution implicitly during generation. Mohammad Golam Sohrab et al. in BiomedCurator: Data Curation for Biomedical Literature observe that sequence-to-sequence extraction easily handles instances where the target information is a normalized version of the source text. Because the model generates text token-by-token, it can output a canonical entity name or database key even if the source text used an abbreviation, a pronoun, or a rephrased expression.