Abstract
Text Expansion (TE) is a novel natural language processing task focused on enriching plain text by seamlessly inserting fine-grained modifiers—such as adjectives, adverbs, and idioms—to make writing more vivid and concrete. This paper presents a comprehensive benchmark for TE, featuring a massive dataset of 12 million pairs across English and Chinese, a novel evaluation metric called Info-Gain, and a robust "Locate & Infill" modeling paradigm. Our experiments demonstrate that while large language models like ChatGPT excel in fluency, specialized "Locate & Infill" models are superior in maintaining semantic integrity and achieving high fidelity.
Introduction: Defining Text Expansion
While modern writing assistants effectively correct grammar or paraphrase sentences, they often lack the capability for creative "polishing"—the addition of descriptive nuances that make human writing engaging. Existing tasks like "Text Infilling" or "Story Elaboration" typically involve fixed insertion points or allow for broad, wandering content.
In contrast, Text Expansion (TE) requires a model to:
- Identify flexible locations where modifiers can be added without disrupting the linguistic flow.
- Generate contextually rich modifiers that enhance the original sentence.
- Preserve core semantics, ensuring the fundamental meaning remains unchanged.
Methods and Metrics
The researchers explored two primary modeling paradigms for TE:
- Text2Text: A standard sequence-to-sequence approach using models like T5, BART, or GPT-2 to transform a source sentence into an expanded version.
- Locate & Infill: A specialized two-step process. First, a model identifies "anchors" (typically nouns or verbs) and places mask tokens around them. Second, an infilling model populates these masks. This can be implemented as a Pipelined system (separate models) or a Joint system (a single model performing both tasks).
To evaluate these models, the authors introduced Info-Gain. Traditional metrics like BLEU or Perplexity often fail to distinguish between useful descriptive expansion and redundant "fluff." Info-Gain utilizes a pre-trained model to quantify how much the added modifiers assist in "recovering" the original context, thereby measuring the specific information added.
Dataset Construction
A massive benchmark was constructed for both English and Chinese, consisting of:
- 12 Million Automatically Generated Pairs: These were created using four strategies: Neural Sentence Compression (NSC) (inverse summarization), Constituency Tree Pruning (CTP) (stripping modifiers via linguistic rules), IsA Relationship (IAR) (extracting descriptive phrases from hypernym patterns), and Masked Modifier Prediction (MMP) (proactively suggesting expansions).
- 2K Human-Annotated Test Set: A high-quality, manually refined set used for rigorous final evaluation.
Key Results and the ChatGPT Paradox
The experimental results highlight a significant performance gap between specialized models and general-purpose LLMs.
The Superiority of Locate & Infill
In both languages, the Locate & Infill models achieved nearly 100% Fidelity, meaning they successfully preserved all original tokens without unauthorized deletions or rewrites. They also outperformed Text2Text baselines on the Info-Gain metric. Interestingly, the Joint model performed better for English, while the Pipelined model was superior for Chinese, likely due to structural differences in how modifiers are positioned in each language.
The ChatGPT Paradox
When evaluated on TE, ChatGPT produced long, fluent expansions. However, it failed significantly on Fidelity (scoring only ~2.4% in English). This is because ChatGPT tends to rewrite the entire sentence rather than strictly inserting modifiers, often changing the original wording and potentially altering the core semantics.
Conclusion and Future Work
This work establishes a solid foundation for Text Expansion. While the proposed models show high fidelity and informativeness, several challenges remain. Current models do not yet check for factual consistency (e.g., ensuring a modifier like "young" is historically accurate) and often default to "safe," generic modifiers. Future research will focus on improving the specificity of expansions and developing more granular evaluation metrics.
Summary updated