PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models

Zheng Zhang, Jinyi Li, Yihuai Lan, Xiang Wang, Hao WangIJCAI 2025 · DOI

Introduction

As Large Language Models (LLMs) are increasingly integrated into complex workflows, the length of input prompts has grown significantly. Techniques such as Retrieval-Augmented Generation (RAG), Few-Shot Learning, and Chain-of-Thought (CoT) often result in massive contexts. This "prompt bloat" presents two primary challenges: economic costs, as proprietary models charge per token, and computational latency, as longer prompts increase processing time.

Prompt compression aims to reduce the number of tokens in a prompt while preserving the essential information required for the LLM to generate an accurate response. However, the field has been fragmented, with various methods requiring specific environments and evaluation metrics. PCToolkit addresses this by providing a modular, unified library that brings these disparate methods under one roof.

The workflow of prompt compression using PCToolkit, showing how an original context is reduced before being processed by an LLM.

Modular Architecture

The core of PCToolkit is its modular design, which allows for a "plug-and-play" experience. The architecture is divided into four main components:

  1. Compressors: The toolkit integrates six state-of-the-art compression methods, categorized into three types: RL-based (e.g., KiS, SCRL), LLM Scoring-based (e.g., Selective Context), and LLM Annotation-based (e.g., the LLMLingua family).
  2. Datasets: A standardized interface for loading various NLP benchmarks.
  3. Metrics: A comprehensive suite of evaluation tools, including BLEU, ROUGE, and BERTScore, to ensure semantic preservation.
  4. Runner: The central engine that orchestrates experiments, allowing users to compare different compressors with minimal code.
The modular architecture of PCToolkit, illustrating the interaction between Datasets, Compressors, Metrics, and the Runner.

Supported Tasks and Datasets

PCToolkit supports a wide range of datasets to evaluate compression across diverse NLP tasks:

  • Reasoning & Mathematics: GSM8K, BBH (Big-Bench Hard).
  • Summarization & Reconstruction: BBC News, Arxiv, Gigaword, DUC2004, BNC, Broadcast, Google.
  • Dialogue & Long Context: ShareGPT, LongBench.

The toolkit provides a mapping between specific tasks (e.g., "Lie Detection" or "Code Completion") and appropriate datasets, ensuring that researchers use the most relevant benchmarks for their specific use cases.

Key Insights

A major contribution of PCToolkit is the abstraction of prompt compression into a single PromptCompressor class. This simplifies the implementation process significantly, turning a complex research task into a straightforward engineering step: result = compressor.compressgo(prompt, ratio).

Furthermore, the toolkit emphasizes the importance of task-specific metrics. For instance, while ROUGE is suitable for summarization, mathematical reasoning requires strict accuracy metrics. By providing a definitive mapping of metrics to datasets, it offers a "gold standard" for evaluating prompt compression research.

Conclusion and Future Work

PCToolkit represents a significant step toward making prompt compression accessible and standardized. It reduces the implementation of complex algorithms from hundreds of lines of custom code to a standardized workflow. Future developments will likely include the integration of newer compression techniques and the implementation of an "Auto-Compressor" that automatically selects the optimal algorithm and compression ratio based on the input task.

The project is open-source and available on GitHub, with a live demo on Hugging Face.

Summary updated

BibTeX
@inproceedings{zhang2025pctoolkit,
  author = {Zheng Zhang and Jinyi Li and Yihuai Lan and Xiang Wang and Hao Wang},
  title = {{PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1277},
}
Figure