TL;DR
This paper tackles the challenge of developing safe chatbots by proposing an automated pipeline to repurpose existing dialogue datasets. By leveraging information retrieval techniques, they effectively revise unsafe responses within datasets, creating safer versions without losing data volume, thus enabling the development of more responsible conversational AI.
What Problem Does This Paper Address?
Large Language Models (LLMs) have significantly advanced the capabilities of open-domain chatbots, making them adept at engaging conversations. However, a critical issue is that these chatbots can generate unsafe responses, ranging from offensive remarks to harmful professional advice. While manually curating or annotating datasets for dialogue safety is an option, it's often expensive and time-consuming. Existing dialogue datasets, often scraped from the internet, can contain toxic or harmful content. This paper aims to bridge this gap by providing an automated method to create safer dialogue datasets, making the development of safe chatbots more accessible and efficient.
Methods
The core of this work lies in an automated pipeline that repurposes existing dialogue datasets for safety. The process involves:
- Dataset Selection: The DIASAFETY dataset is chosen as the primary source for identifying unsafe utterances due to its annotated safe and unsafe examples across various categories.
- Dataset Annotation (Automated): Instead of manual annotation, the authors fine-tune a RoBERTa-base classifier to automatically label examples as safe or unsafe. This classifier is trained on a subset of the DIASAFETY dataset and uses a strict condition (predicting safe for both response-only and context-response pair inputs) to minimize false negatives.
- Response Selection (Information Retrieval): For identified unsafe responses, the system retrieves safer alternatives from a collection of safe responses within the same dataset. Two primary IR techniques are employed:
- BM25: A sparse retrieval algorithm that relies on keyword matching and term frequency to find relevant sentences.
- Sentence Transformers (SBERT): A dense retrieval method that uses embeddings to capture semantic similarity between the user's input (query) and potential safe responses.
- Dataset Modification: The original unsafe responses are replaced with the top-ranked safe responses retrieved in the previous step. This process ensures that the modified dataset retains the same number of examples as the original, avoiding data reduction.
Datasets
- DIASAFETY: This dataset is used for identifying and revising unsafe utterances. It's a labeled dataset of over 11,000 examples, categorized into five types of unsafe content: Unauthorized Expertise, Toxicity Agreement, Risk Ignorance, Biased Opinion, and Offending User.
- ParlAI Framework Datasets: Several other dialogue datasets available on the ParlAI framework were used for model development and fine-tuning. These include:
- ConvAI2: A large crowdsourced dataset (over 140k utterances) extending PersonaChat.
- Wizard of Wikipedia: Contains sentences from 5.4M Wikipedia articles, used for chit-chat.
- EmpatheticDialogues: A dataset of over 25k emotionally grounded conversations.
- BlendedSkillTalk: A crowdsourced dataset of about 5k conversations designed to blend skills like personality, knowledge, and empathy.
These latter datasets were not modified by the authors' approach, as they were curated with safety in mind.
Key Insights and Ideas
- Automated Safety Annotation: The successful fine-tuning of a classifier for automatic safety annotation significantly reduces the reliance on expensive human annotation.
- IR for Safety Repurposing: Information retrieval techniques, particularly dense methods like SBERT, can effectively find semantically relevant and safe responses to replace unsafe ones, even when lexical overlap is minimal.
- Preserving Dataset Size: Replacing unsafe responses rather than filtering them out is a crucial strategy for maintaining the volume of training data, which is vital for robust model development.
- SBERT's Semantic Advantage: SBERT-based retrieval shows a stronger ability to find contextually relevant safe responses compared to BM25, which relies more on exact keyword matching. This highlights the importance of semantic understanding in dialogue safety.
Key Results
- Automatic Evaluation: Models fine-tuned on the BM25-modified dataset achieved the highest F1 score in the "Safe" class (0.92), while the original DIASAFETY dataset had the highest F1 score in the "Unsafe" class (0.81).
- Human Evaluation: Raters found the BM25 and SBERT modified datasets to contain fewer unsafe examples than the original. The SBERT modified dataset showed highly competitive results in terms of relevance. While BM25 modification resulted in fewer unsafe examples, it was rated as less contextually relevant.
- Model Performance: Models fine-tuned on the modified datasets (Ft+BM25 and Ft+SBERT) generated significantly fewer unsafe responses compared to the model fine-tuned on the original DIASAFETY dataset (Ft+DiaSafety). Specifically, Ft+BM25 and Ft+SBERT achieved 0.0 F1 score for unsafe responses, while Ft+DiaSafety had 0.18.
Limitations and Future Work
- Language Limitation: The current work is limited to English language dialogue datasets.
- Future Work: The authors suggest exploring the application of their approach to dialogue datasets in under-resourced languages. They also emphasize that their technique is intended for detoxifying dialogue models, not for making them more toxic. Further research could investigate the effectiveness of their approach on dialogue datasets in under-resourced languages.
Summary updated