TL;DR
This paper introduces a comprehensive benchmark for evaluating how well AI-generated legal answers are supported by provided source material. They developed a new dataset and tested various methods, finding that while some advanced prompting techniques achieve high accuracy, they often come with significant latency, highlighting a crucial trade-off for real-world applications.
What Problem Does This Paper Address?
Generative AI systems are increasingly used in high-stakes domains like legal question-answering. A critical challenge is ensuring these systems provide accurate and trustworthy responses that are "grounded" in the provided legal context. Ungrounded or "hallucinated" responses can erode user trust and diminish the utility of these AI tools. Existing methods for assessing this groundedness vary, and there's a need for a systematic comparison of these approaches, especially within the complex and nuanced legal domain. This paper aims to fill this gap by benchmarking different methods for detecting ungrounded legal AI responses.
Methods
The authors explored three main categories of methods for assessing groundedness:
- Similarity-based techniques: These methods quantify how well generated text aligns with the source material at a sentence level. They specifically used:
- Semantic Similarity: Employing a legal BERT model to embed sentences and using cosine similarity to match pairs, with an optimized threshold.
- Quoted Information Precision (QuIP): Adapting an existing metric to measure character n-gram overlap between responses and context, also with optimized parameters.
- Natural Language Inference (NLI) models: These models determine if sentences in the generated response are entailed by or contradict sentences in the source material. They used:
- FactKB: A model leveraging pre-trained facts to evaluate factual consistency.
- Hallucination Evaluation Models (HEM): Specifically, HEM V1 (based on DeBERTa) and HEM V2 (based on Flan-T5), which classify summaries as factually consistent with their source.
- Prompting Approaches: These methods use LLMs themselves to classify responses. They tested:
- Direct Prompting: Asking LLMs (Lynx-v1.1, GPT-40, Claude Sonnet 3.5) directly if a response is grounded.
- Amazon RefChecker: A multi-stage approach involving claim extraction, hallucination checking, and aggregation.
- SelfCheckGPT: An adaptation that generates multiple responses to assess consistency, enhanced with a novel context-based evaluation (ContextNLI).
- DeepEval Claims Verify: A multi-LLM call approach for claim extraction and verification.
They also explored fine-tuning a Cross-Encoder classifier (DeBERTa v3) on a specialized dataset to capture nuanced differences between grounded and ungrounded responses.
Datasets
The primary dataset used in this study was newly created and named the Groundedness Classification (GC) dataset.
- Source: Proprietary data from Casetext Legal Research Skill, consisting of legal queries, LLM-generated responses, and retrieved context.
- Ground Truth: Responses were initially verified by legal professionals for groundedness.
- Synthetic Adaptation: To create ungrounded examples, GPT-40 was instructed to introduce subtle, non-intrusive deviations from the provided context into the original grounded responses.
- Splits: The dataset was split into training (70%), development (10%), and testing (20%) sets. Table 1 provides statistics on the number of queries, responses, and response sentences per split.
Key Insights and Ideas
- No Single Best Method: The study reveals a clear trade-off between accuracy (F1 score) and computational efficiency (latency). Top-performing methods are often slower, while faster methods are less accurate.
- Advanced Prompting Shows Promise: Multi-stage prompt chaining approaches like DeepEval Claims Verify achieved the highest F1 scores, indicating the power of carefully crafted prompts for complex tasks.
- Latency is a Major Hurdle: While DeepEval Claims Verify was highly accurate, its latency of 26.1 seconds per request makes it impractical for real-time applications. Direct prompting with GPT-40 offered a better balance, with high scores and significantly lower latency.
- Similarity Methods are Fast but Limited: Similarity-based approaches were the fastest but struggled to accurately identify ungrounded responses.
- Factual Inaccuracies Dominate Errors: The error analysis identified "Factual Inaccuracies" as the most prevalent type of ungrounded content, followed by "Reasoning Errors."
- Fine-tuning for Specificity: Fine-tuning a DeBERTa v3 model on the legal dataset showed promising results, particularly for distinguishing subtle differences in groundedness.
Key Results
- Top Performer: DeepEval Claims Verify achieved the highest F1 score of 0.80 on the test set.
- Best Latency/Accuracy Balance: Direct prompting with GPT-40 achieved a strong F1 score of 0.77 with a much lower latency of 2.2 seconds.
- Similarity Method Performance: COS_SIM achieved a macro-F1 of 0.497 with very low latency (1.2s), while QUIP achieved 0.509 with 1.5s latency.
- NLI Model Performance: HEM V1 achieved an F1 of 0.595, and HEM V2 achieved 0.563. FactKB scored 0.510.
- Error Analysis: Factual Inaccuracies constituted 78.67% of unique error types in the development set.
- Misclassification: For the best model (DeepEval Claims Verify), Terminological Errors had the highest misclassification rate (67%), despite their low frequency.
Limitations and Future Work
- Dataset Size and Scope: The current dataset is limited in size and domain, potentially affecting the generalizability of findings to other models or legal sub-domains.
- Binary Classification: Classifying responses as strictly "grounded" or "ungrounded" might oversimplify reality, as responses can contain mixed elements.
- Computational Resources: Some complex methods require significant computational resources, limiting their immediate practical applicability.
- Model Bias: Error classification relies on agreement between specific LLMs, which could introduce biases.
- Rapid Model Evolution: The fast pace of LLM development means findings may quickly become outdated.
Future work should focus on:
- Expanding the dataset to include more domains and increasing its size.
- Developing more nuanced classification frameworks that capture mixed groundedness.
- Exploring scalable, real-time methods suitable for resource-constrained environments.
- Continuously updating evaluation frameworks to keep pace with LLM advancements.
Summary updated