Abstract
In this paper, we introduce the new task of controllable text edition, in which we take as input a long text, a question, and a target answer, and the output is a minimally modified text, so that it fits the target answer. This task is very important in many situations, such as changing some conditions, consequences, or properties in a legal document, or changing some key information of an event in a news text. This is very challenging, as it is hard to obtain a parallel corpus for training, and we need to first find all text positions that should be changed and then decide how to change them. We constructed the new dataset WikiBioCTE for this task based on the existing dataset WikiBio. We use WikiBioCTE for training, and manually labeled a test set for testing. We also propose novel evaluation metrics and a novel method for solving the new task. Experimental results on the test set show that our proposed method is a good fit for this novel NLP task.
Introduction
Updating documents—such as legal contracts, news articles, or biographies—is a common but labor-intensive task. While modern language models can generate text from scratch, they often lack the precision required for surgical edits. We define this challenge as Controllable Text Edition (CTE): given a document $D$, a question $Q$, and a target answer $A'$, the goal is to produce a modified document $D'$ that reflects the new information while remaining fluent and minimally changed from the original.
The challenge in CTE is twofold. First, Localization: the model must identify exactly which parts of a long document serve as the "rationale" for a specific answer. Second, Fluency & Minimality: the edit should not be a simple "find and replace," which often breaks grammar, nor a total rewrite, which discards valuable context. Furthermore, the field suffers from a lack of parallel data (original vs. edited versions) for supervised training.
Methodology: The Select-Mask-Generate (SMG) Framework
To address these challenges, we propose the Select-Mask-Generate (SMG) framework. This approach decomposes the editing process into three distinct phases:
- Selector: Utilizing a BiLSTM with a Gumbel Softmax, the model identifies "answer-related" tokens in the original document $D$. This allows the selection process to remain differentiable during training.
- Masking: The identified tokens are replaced with masks, creating a "context template" that preserves the non-target information.
- Partial Generation: Instead of a standard decoder generating a full sequence, the SMG decoder fills the masked blanks. It employs a Copy Mechanism to draw from the original text or the new answer and introduces an End-of-Answer (EOA) tag. This tag signals the model to stop generating and resume reading the existing template, allowing for edits of arbitrary length without disrupting the surrounding structure.
Training via Reconstruction
Because parallel "before and after" data is scarce, we utilize a reconstruction task for training. The model learns to reconstruct the original document $D$ using the original answer $A$. At inference time, we swap the original answer $A$ for the target answer $A'$, and the model generates the modified document $D'$.
Datasets
We introduce WikiBioCTE, a dataset derived from the WikiBio corpus of Wikipedia biographies. We treat infobox fields (e.g., "Occupation") as questions and their corresponding values (e.g., "Virologist") as answers.
- Training Set: 141k triples of (Document, Question, Answer).
- Test Set: Approximately 1,000 samples manually annotated by linguists to provide ground-truth modified documents ($D'$) for rigorous evaluation.
Results and Discussion
The SMG model significantly outperforms standard Sequence-to-Sequence (Seq2Seq) baselines across multiple dimensions:
- Correctness: In human evaluations, SMG achieved 80.2% correctness in successfully updating the answer, compared to 73.5% for the baseline.
- iBLEU: SMG scored 10.05, nearly doubling the baseline's 5.63, indicating superior performance in making necessary changes while preserving the original text.
- diff-BLEU ratio: This metric specifically measures the quality of the edited segments. SMG reached 62.5%, while the baseline lagged at 21.3%.
Qualitatively, SMG avoids the common pitfalls of standard generation, such as "hallucinations" or the accidental blending of old and new information.
Conclusion
The SMG framework provides a robust solution for the CTE task by combining selective masking with partial generation. While the model is highly effective, future work will address selector errors—where missed pronouns can lead to contradictory information—and explore more complex domains like legal or medical texts where rationales may be non-contiguous.
Summary updated