When we summarize a document, we often face a strict word budget. The simplest way to save space is to delete words, but deleting words at random quickly turns a coherent thought into "word salad." To shorten a sentence while keeping it readable, we must respect its underlying architecture.
Researchers have converged on a powerful solution: treating sentence compression as a structural pruning problem on a linguistic parse tree. By identifying the "grammatical skeleton" of a sentence, systems can discard modifiers and redundant clauses while ensuring the remaining text remains a valid, logical unit.
The Sentence as a Tree
The core intuition behind this research is that sentences are not linear strings of tokens, but hierarchical structures. Most work in this area, such as that by Liqun Shao in the DTATG system, relies on dependency trees. In these trees, words are nodes, and edges represent grammatical relationships (like a subject belonging to a verb).
If you delete a word in the middle of a string, you might break the sentence. But if you prune a "leaf" or a sub-branch from a dependency tree, the rest of the tree remains connected. Peng Li and Yinglin Wang demonstrated that pruning based on these relations—such as removing clausal complements or parataxis—ensures that the compressed text remains grammatically readable compared to simple word-level deletion [SJTU CIT at TAC 2010].
Preserving the Grammatical Core
How do we decide what to keep? The consensus across these papers is to prioritize the Subject-Verb-Object (SVO) core. Hongyan Liu and colleagues describe this as identifying the "grammatical core" through dependency parsing to prune extraneous modifiers while maintaining the primary meaning [The CIST Summarization System at TAC 2010].
However, a "one-size-fits-all" pruning rule can be too rigid. Different contexts require different levels of detail:
- Summarization Density: David N. Milne and Vivi Nastase used tree pruning to maximize the density of "summary-worthy" concepts within a limited word budget [Summarizing with Encyclopedic Knowledge].
- Linguistic Variation: In Japanese, Shun Hasegawa and Hiroya Takamura found that constraints must be relaxed to allow non-verbal elements (like nouns in headlines) to serve as the root of the compressed sentence [Japanese Sentence Compression with a Large Training Dataset].
- Semantic Preservation: Chen Li and Yang Liu argued for operating on nodes within hierarchical parse trees rather than individual words, ensuring that entire syntactic units (like prepositional phrases) are kept or removed as a single coherent block [Improving Multi-documents Summarization by Sentence Compression].
Head-Driven and Weighted Pruning
To prevent "orphaned" words—modifiers that remain after their head word has been deleted—researchers like Lu Wang developed head-driven pruning. This approach links the removal of modifiers to their respective head nodes in the syntax tree, ensuring that if a "branch" is cut, all its "twigs" go with it [A Sentence Compression Based Framework to Query-Focused Multi-Document Summarization].
We can also guide the pruning process using external "importance" signals. Zhongyu Wei and Wei Gao showed that by adjusting the weights of edges in a dependency tree based on real-time social media signals (like tweets), a model can be "nudged" to retain specific entities that are currently trending, even if they aren't the primary grammatical subject [Using Tweets to Help Sentence Compression].
From Summaries to LLM Prompts
While early work focused on news summarization, the same structural intuition is now being applied to Large Language Models (LLMs). As context windows become a bottleneck, researchers are looking for ways to compress long prompts without losing the "human logic" required for reasoning.
The PartPrompt system by Wenhao Mao and Ke Tang treats prompt compression as a tree-pruning problem. By using linguistic parse trees to guide the removal of tokens, they prevent the semantic fragmentation that occurs with unstructured deletion. This ensures that the logical flow of the instructions remains intact, allowing the model to reason effectively even with a fraction of the original text [Parse Trees Guided LLM Prompt Compression].