TL;DR
This paper introduces a novel framework for generating citation sentences that allows users to control specific attributes like citation intent and keywords. By combining supervised fine-tuning with reinforcement learning, they achieve more controllable and accurate citation generation, outperforming existing methods and even large language models like GPT-3.5.
What Problem Does This Paper Address?
Scientific writing relies heavily on citations to support arguments, provide background, or compare results. However, existing automated citation generation systems often lack user control, forcing authors to accept generated sentences that may not align with their specific intentions or desired keywords. This gap limits the flexibility and precision authors need when integrating citations into their work. The paper addresses this by developing a system that allows users to explicitly guide the citation generation process.
Methods
The authors propose a two-phase approach:
- Supervised Fine-tuning: They design a structured input template that incorporates manuscript context, cited paper information, and user-specified citation attributes (intent and keywords). This template is used to fine-tune a language model (LM) via next-token prediction. This phase teaches the LM to generate sentences conditioned on these attributes.
- Controllability Enhancement via Reinforcement Learning: To further improve controllability, they employ Proximal Policy Optimization (PPO). They define several controllability metrics (Intent Alignment Score, Keyword Recall, Fluency Score, and ROUGE-F1) which are used to create a reward function for PPO. This allows the LM to directly optimize for adherence to user-specified attributes.
Datasets
The authors curated a dataset from arXiv computer science papers. This dataset consists of triplets: the citing paper (manuscript), the citation sentence within it, and the cited paper. They extracted contextual information and used SciBERT to predict citation intents and Sentence-BERT to extract keywords for each citation. The dataset was split chronologically to prevent data leakage.
Key Insights and Ideas
- Controllability is Crucial: The paper strongly emphasizes that explicit user control over citation generation is not just desirable but necessary for practical scientific writing.
- PPO for Fine-Grained Control: Reinforcement learning, specifically PPO, is highly effective in fine-tuning LMs to adhere to specific citation attributes, leading to significant improvements in controllability metrics.
- Attribute Synergy: Combining intent and keyword control yields better results than controlling only one attribute, demonstrating the power of multi-attribute guidance.
- Pretraining Matters: LMs pretrained on scientific corpora (like Galactica) perform significantly better in citation generation tasks compared to those pretrained on more general datasets (like GPT-Neo), highlighting the importance of domain-specific knowledge.
Key Results
- The proposed controllable citation generation framework, especially when enhanced with PPO, significantly improves controllability metrics (Intent Alignment Score, Keyword Recall, Fluency Score) compared to uncontrolled generation.
- Models like Galactica-6.7B-PPO, after PPO fine-tuning, achieved superior performance across multiple metrics, including ROUGE-F1 scores, outperforming even large models like GPT-3.5-turbo in human evaluations.
- Human evaluations confirmed that while GPT-3.5-turbo can be fluent, it often struggles with contextually accurate citation generation and intent alignment compared to the fine-tuned models.
- The study found that larger beam sizes in generation do not necessarily improve performance and can sometimes negatively impact ROUGE scores.
Limitations and Future Work
- Model Size and Pretraining: While larger models and scientific pretraining are beneficial, the paper notes that performance can still vary between models of similar size, suggesting other factors are at play.
- Prompt Engineering for Large Models: For very large models like GPT-3.5-turbo, further prompt refinement or few-shot tuning might be needed to fully leverage their capabilities for controllable citation generation.
- Future Research: The authors suggest deeper exploration into the influence of beam size on citation generation performance and further research into optimizing the balance between automation and controllability.
Summary updated