TL;DR
LargePiG is a training-free, plug-and-play decoding method that transforms Large Language Models (LLMs) into Pointer-Generator networks to mitigate hallucinations during query generation. By leveraging internal attention weights to copy factual tokens directly from the input and using early-exit layer differences to decide when to copy, it significantly improves query relevance and factuality with minimal computational overhead.
What Problem Does This Paper Address?
Query generation—the process of automatically creating search queries from documents or videos—is essential for modern search engines and short-video recommendation platforms. However, when LLMs are tasked with generating these queries, they frequently suffer from two distinct types of hallucinations:
- Factuality Hallucinations: Generating queries containing false facts or details that contradict the source material.
- Relevance Hallucinations: Generating queries that are factually true in a general sense but completely irrelevant to the specific input context.
Traditional methods to fix these issues rely on expensive model retraining or complex post-processing retrieval steps. This paper addresses the challenge at the decoding source by decoupling content (factual tokens extracted directly from the input) from form (the syntactic structure generated by the LLM).
Methods
The authors introduce LargePiG (Large Pointer-Generator), a model-agnostic, training-free plug-in that forces an LLM to act as a pointer-generator network using purely internal, pre-trained mechanisms:
- Pointer Attention Distribution: Instead of training a separate pointer network to locate source words, LargePiG extracts the LLM's internal self-attention weights from the last generated token back to the source tokens.
- Vocabulary Distribution: It retains the standard next-token probability distribution from the final layer to maintain linguistic fluency and grammatical structure.
- Copy Probability ($p_{copy}$): To dynamically decide whether to copy a word from the text or generate a new one, LargePiG compares the vocabulary distributions of the model's high layers (via early exiting) against the final anchor layer.
Datasets
To rigorously evaluate their approach, the authors introduced two new benchmarks specifically tailored for truthful query generation:
- TruthfulVQG: A Chinese dataset collected from a real short-video platform containing video titles, dialogue (ASR), and on-screen text (OCR).
- TruthfulDQG: An English document-based dataset adapted from the MS-MARCO machine reading comprehension dataset.
- TruthfulVQG-M: A multimodal variant used to test Large Vision-Language Models (LVLMs) using video covers.
They also validated copy mechanics using the standard SQuAD dataset and general factuality via the FACTOR benchmark.
Key Insights and Ideas
- Attention is Truthful, FFNs Hallucinate: The authors note that factual knowledge is primarily stored within the Feed-Forward Network (FFN/MLP) modules of an LLM. These modules can override the input context with incorrect pre-trained biases. Conversely, self-attention modules remain highly focused on the provided context, making them excellent "pointers."
- Linguistic Complexity Signals Copying: A key discovery is that LLM predictions for simple function words (prepositions, conjunctions, auxiliary verbs) stabilize early in the higher layers of the network. In contrast, complex factual words (names, places, dates) continue to evolve until the very last layer. LargePiG exploits this distributional delta to dynamically calculate the copy probability without adding any extra parameters.
Key Results
- Across-the-Board Improvements: LargePiG consistently outperformed base decoding and state-of-the-art contrastive decoding methods (like DoLa) across multiple models (Qwen1.5-7B and LLaMA2-7B) on both benchmarks.
- Multimodal Success: When applied to LLaVA-7B and LLaVA-13B, LargePiG successfully boosted truthfulness metrics (MC1, MC2, MC3) by mapping image tokens to text tokens in the embedding space.
- Enhanced Copying on Long Inputs: On the SQuAD dataset, LargePiG significantly improved F1 scores, with the most pronounced gains occurring on long context inputs where traditional LLMs typically struggle with copy fidelity.
- Negligible Latency: The method introduces a maximum decoding latency increase of only 6% to 10%, proving its practical viability for production environments.
Limitations and Future Work
- Scale Constraints: Due to limited computing resources, empirical validation was primarily restricted to 7B-parameter models. Verifying the effects of LargePiG on larger scales (e.g., 70B+ models) remains future work.
- Layer Selection Optimization: Currently, LargePiG relies on a heuristic selection of the last layer's attention weights. The authors suggest that exploring supervised probing techniques could optimize layer selection for even better pointer distributions.
- RAG Integration: A promising future direction is directly applying LargePiG to Retrieval-Augmented Generation (RAG) pipelines to suppress downstream hallucinations when synthesizing retrieved context.
Summary updated