VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models

Ming Cheng, Jiaying Gong, Chenhan Yuan, William A. Ingram, Edward A. Fox, Hoda Eldardiry

Abstract

Existing text simplification or paraphrase datasets mainly focus on sentence-level text generation in a general domain. These datasets are typically developed without using domain knowledge. In this paper, we release a novel dataset, VTechAGP, which is the first academic-to-general-audience text paraphrase dataset consisting of document-level theses and dissertation academic and general-audience abstract pairs from 8 colleges authored over 25 years. We also propose a novel dynamic soft prompt generative language model, DSPT5. For training, we leverage a contrastive-generative loss function to learn the keyword vectors in the dynamic prompt. For inference, we adopt a crowd-sampling decoding strategy at both semantic and structural levels to further select the best output candidate. We evaluate DSPT5 and various state-of-the-art large language models (LLMs) from multiple perspectives. Results demonstrate that the SOTA LLMs do not provide satisfactory outcomes, while the lightweight DSPT5 can achieve competitive results. To the best of our knowledge, we are the first to build a benchmark dataset and solutions for academic-to-general-audience text paraphrase dataset.

Introduction

Scientific communication often hits a wall known as the "ivory tower" effect: research is written by experts, for experts, in a dialect of jargon that is nearly impenetrable to the public or even researchers in adjacent fields. While "text simplification" is a known AI task, most existing datasets (like WikiAuto or Newsela) focus on sentence-level changes or are designed to simplify text for children. There is a massive gap in tools that can handle document-level paraphrasing—taking a full, technical abstract and rewriting it for a "lay" audience without losing the scientific soul of the work. This paper introduces VTechAGP, a high-quality benchmark, and DSPT5, a model designed to overcome the tendency of general-purpose LLMs to either hallucinate details or fail to grasp specific domain nuances when simplifying technical text.

The VTechAGP Dataset

The core of this research is the introduction of VTechAGP, containing 4,938 pairs of academic and general-audience abstracts. These were harvested from Electronic Theses and Dissertations (ETDs) at Virginia Tech spanning 25 years. Unlike previous datasets, VTechAGP is document-level and covers eight distinct colleges (from Veterinary Medicine to Architecture), providing a rich diversity of technical domains. It also includes auxiliary metadata like discipline, degree level, and author-provided keywords.

Methodology: The DSPT5 Model

The authors propose a new model called DSPT5, built on the relatively small T5 architecture (220 million parameters). Its effectiveness stems from three main components:

  1. Dynamic Soft Prompting: Instead of using a static instruction (like "Simplify this"), the model uses KeyBERT to extract specific technical keywords from the source text. These are transformed into "soft prompts"—learnable mathematical vectors that help the model adapt its strategy based on the specific academic domain.
  2. Hybrid Loss Function: During training, the model uses a combination of standard cross-entropy loss and contrastive loss (infoNCE). This forces the model to learn the difference between technical "expert" keywords and general-audience language in a shared feature space.
  3. Crowd-Sampling Decoding: At the inference stage, the model generates multiple candidate summaries and selects the best one by measuring semantic and structural alignment using metrics like BERTScore and Levenshtein similarity.
The DSPT5 architecture featuring dynamic soft prompting and contrastive-generative loss.

Experimental Results

The researchers compared DSPT5 against heavyweights like ChatGPT, LLaMA2, and Claude2 using both automated metrics and human evaluation.

Comprehensive performance comparison of DSPT5 and SOTA LLMs across multiple metrics and colleges.

As shown in the results, DSPT5 achieved the highest scores in s-BLEU (12.41) and ROUGE1 (52.71), significantly outperforming ChatGPT (s-BLEU: 4.70) and LLaMA2 (s-BLEU: 3.21) in terms of similarity to the "gold standard" lay summaries. Human evaluation further validated these findings: in a 1-5 Likert scale test, DSPT5 tied with ChatGPT for the highest Comprehensiveness score (4.17) and beat all other models in Meaning Preservation (3.90).

Conclusion

DSPT5 demonstrates that for niche tasks like academic-to-lay paraphrasing, high-quality, task-specific data is more valuable than raw model scale. Being lightweight enough to be fine-tuned on a single V100 GPU, DSPT5 offers a far more accessible and secure option for research institutions than relying on expensive, closed-source APIs.

Summary updated

Abstract

Existing text simplification or paraphrase datasets mainly focus on sentence-level text generation in a general domain. These datasets are typically developed without using domain knowledge. In this paper, we release a novel dataset, VTechAGP, which is the first academic-to-general-audience text paraphrase dataset consisting of document-level these and dissertation academic and general-audience abstract pairs from 8 colleges authored over 25 years. We also propose a novel dynamic soft prompt generative language model, DSPT5. For training, we leverage a contrastive-generative loss function to learn the keyword vectors in the dynamic prompt. For inference, we adopt a crowd-sampling decoding strategy at both semantic and structural levels to further select the best output candidate. We evaluate DSPT5 and various state-of-the-art large language models (LLMs) from multiple perspectives. Results demonstrate that the SOTA LLMs do not provide satisfactory outcomes, while the lightweight DSPT5 can achieve competitive results. To the best of our knowledge, we are the first to build a benchmark dataset and solutions for academic-to-general-audience text paraphrase dataset. Models will be public after acceptance.
BibTeX
@inproceedings{cheng2025vtechagp,
  author = {Ming Cheng and Jiaying Gong and Chenhan Yuan and William A. Ingram and Edward A. Fox and Hoda Eldardiry},
  title = {{VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=8jFwYDC4Dt},
}
Figure