TL;DR
This thesis proposal tackles the critical issue of reliability in Natural Language Generation (NLG) systems, particularly large language models (LLMs). It proposes novel methods for evaluating semantic accuracy and exploring interpretability techniques to pinpoint and mitigate inaccuracies, aiming to make NLG systems more trustworthy.
What Problem Does This Paper Address?
The rapid rise of powerful Large Language Models (LLMs) has brought Natural Language Generation (NLG) to the forefront, with millions now interacting with these systems daily. However, a significant challenge remains: ensuring the reliability of their outputs. Unlike humans who can admit ignorance, LLMs often "hallucinate" plausible-sounding but incorrect information. This can lead to user distrust and, in critical scenarios, the spread of misinformation. The paper argues that while fluency is important, semantic accuracy—the faithful representation of underlying input data—is paramount for reliability. Current evaluation metrics often fall short, lacking interpretability or requiring extensive human annotation. This thesis aims to bridge this gap by developing methods to rigorously evaluate and improve the semantic accuracy of NLG systems.
Methods
The proposed research is divided into two main parts:
- Semantic Accuracy Evaluation: The authors propose a novel, reference-less method for evaluating semantic accuracy. This method aims to align portions of the input data (e.g., semantic triples) with phrases in the generated text. Omissions occur when data cannot be aligned, and hallucinations are flagged when text phrases cannot be aligned to any data. This approach leverages dependency trees for phrase-level granularity and is inspired by existing work on alignment and error detection. A key goal is to create a synthetic dataset for evaluating NLG metrics, automatically generating examples of accurate texts, extrinsic hallucinations, intrinsic hallucinations, and omissions.
- Mitigating Inaccuracies with Interpretability: This part focuses on understanding why inaccuracies occur within LLMs. The researchers plan to use:
- Attention Visualization: To identify patterns in attention mechanisms that correlate with hallucinations.
- Probing: To extract information about graph structures (entities and relations) from hidden layers, aiming to pinpoint where and how hallucinations are introduced or input data is forgotten.
- Pruning: Based on insights from attention visualization and probing, they aim to identify and remove specific attention heads that contribute to inaccuracies.
- Fine-tuning: Utilizing efficient methods like LoRA and QLoRA, they plan to fine-tune LLMs on available data-to-text datasets to improve semantic accuracy.
- Modelling Uncertainty: Exploring Bayesian methods to estimate model confidence and encourage systems to admit uncertainty rather than hallucinate.
- Prompt Engineering: Investigating how different prompting strategies (zero-shot, few-shot, chain-of-thought) influence semantic accuracy.
Datasets
The paper mentions the following datasets and their relevance:
- Existing Data-to-Text Datasets: Datasets like WebNLG, Enriched WebNLG, and DART are mentioned as sources for positive examples (accurate texts). However, they are noted as insufficient for benchmarking evaluation metrics due to a lack of explicit hallucination annotations.
- Thomson and Reiter (2021) Dataset: This dataset is highlighted as being close to the research goals, intended for error detection in table-to-text generation. It contains high-quality human annotations but is small (90 examples).
- Maynez et al. (2020) Dataset: Created for abstractive summarization, this dataset is mentioned as a related effort but not directly suitable for data-to-text evaluation.
- Dziri et al. (2022) Dataset: A similar dataset for dialogue systems is noted, but the task is outside the scope of this thesis.
- Synthetic Dataset: A significant contribution will be the creation of a new, larger dataset for data-to-text metric evaluation, generated automatically by dropping semantic triples and sentences. This dataset will include examples of accurate texts, extrinsic and intrinsic hallucinations, and omissions.
- LLMs with Openly Accessible Weights: The research will utilize models like Llama, BLOOM, and others for experiments.
Key Insights and Ideas
- Semantic Accuracy as the Cornerstone of Reliability: The paper strongly advocates for semantic accuracy as the primary driver of trust in NLG systems, moving beyond mere fluency.
- Dual Approach: Outside-In and Inside-Out: The research strategy combines external evaluation (how well does the output match the data?) with internal interpretability (why did the model produce this output?).
- The Importance of Explainability in Evaluation: The proposed evaluation method aims to be explainable, not just providing a score but identifying where inaccuracies occur, aiding users in fact-checking.
- Automated Dataset Generation for Scalability: Recognizing the cost of human annotation, the proposal emphasizes creating a large, synthetic dataset for robust metric evaluation.
- Interpretability for Targeted Mitigation: The core idea is that understanding the internal workings of LLMs (via attention, probing) is crucial for developing effective strategies to reduce inaccuracies.
Key Results
As this is a thesis proposal, concrete results are not yet available. The paper outlines the intended contributions:
- A novel reference-less evaluation method for semantic accuracy in data-to-text generation.
- A proposed method for constructing a larger, synthetic dataset for evaluating NLG metrics focused on semantic accuracy.
- Insights into the sources of semantic inaccuracies within LLMs through interpretability techniques.
- Potential mitigation strategies for improving semantic accuracy, including pruning, fine-tuning, and uncertainty modeling.
Limitations and Future Work
- Potential for Training Data Contamination: The authors acknowledge the possibility that LLMs might have been trained on parts of the datasets used for evaluation. They plan to use data extraction techniques to verify this.
- Difficulty in Finding New Unseen Data: Identifying truly novel data for evaluation will be challenging and requires community effort.
- Interpretability Challenges: Attention visualization can be hard to interpret and potentially misleading.
- Future Work: The proposal naturally leads to future work in developing more robust evaluation metrics, creating larger and more diverse datasets, and applying these methods to a wider range of NLG tasks and models. The research also opens avenues for exploring the ethical implications of unreliable NLG systems.
Summary updated