TL;DR
This paper introduces a novel method for assessing the mental health of chatbots, moving beyond traditional metrics like fluency and relevance. They found that popular open-domain chatbots exhibit significant issues across depression, anxiety, and empathy, suggesting a critical oversight in current development practices.
What Problem Does This Paper Address?
As chatbots become increasingly integrated into our daily lives, their potential to influence users' mental well-being is a growing concern. Current chatbot evaluation metrics focus on technical aspects like response quality, but fail to address the crucial dimension of a chatbot's "mental health." This is particularly problematic for chatbots interacting with vulnerable populations like minors. The paper highlights a critical gap: the lack of a systematic way to assess whether chatbots exhibit negative psychological tendencies (e.g., depression, anxiety) or lack positive ones (e.g., empathy), which could negatively impact users.
Methods
The core contribution is a questionnaire-based mental health assessment method for chatbots. This method adapts established psychological questionnaires used for human assessment into conversational prompts. The pipeline involves:
- Questionnaire Rewriting: Original psychological questions are transformed into natural language utterances suitable for chatbot interaction, including instructions on response format.
- Inquiry with Chatbots: Chatbots are queried using these rewritten questions, employing both single-turn and multi-turn conversational strategies to collect responses.
- Response-Option Alignment: Chatbot responses are manually aligned with the predefined answer options of the original questionnaires. A "Failure" option is introduced for irrelevant or uninterpretable responses.
- Severity Evaluation: Aligned responses are scored based on the questionnaire's rating scale, and a default value is used for failed responses to calculate overall scores and determine the severity of mental health issues.
Datasets
The paper does not introduce new datasets for training. Instead, it utilizes existing, well-known open-domain chatbots that were trained on various large-scale dialogue datasets. These include:
- Blender (2.7B version): Pre-trained on Reddit and fine-tuned with BST (Blended Skill Talk, Wizard of Wikipedia, ConvAI2, Empathetic Dialogues).
- DialoGPT (762M version): Fine-tuned with BST.
- Plato (1.6B version): Pre-trained on Reddit and fine-tuned with BST.
- DialoFlow: Pre-trained on Reddit and fine-tuned with BST.
The evaluation relies on established psychological questionnaires: PHQ-9 (Depression), GAD-7 (Anxiety), CAGE (Alcohol Addiction), and TEQ (Empathy).
Key Insights and Ideas
- Chatbots have "mental health" issues: The most striking finding is that all tested chatbots exhibit severe mental health problems, including depression, anxiety, and a lack of empathy.
- Current development practices are insufficient: The authors posit that these issues stem from the neglect of mental health risks during dataset curation and model training.
- Standard evaluation metrics are inadequate: Traditional metrics like fluency and relevance do not capture the critical aspect of a chatbot's psychological impact on users.
- Questionnaire-based assessment is feasible: Adapting human psychological assessments to chatbots provides a quantifiable and systematic way to evaluate their mental health.
- Single-turn vs. Multi-turn: While trends are generally similar, multi-turn inquiries sometimes reveal more severe issues (e.g., DialoFlow's anxiety), suggesting that conversational history can influence chatbot responses.
- Plato's robustness: Plato showed the least variation in scores across questions and inquiry strategies, indicating a more stable (though still problematic) mental health profile.
Key Results
The experimental results, summarized in Table 2, reveal concerning trends:
- Depression: Scores ranged from 14.09 to 18.60, indicating moderate to moderately severe depression across all chatbots.
- Anxiety: Most chatbots scored above 10 (moderate anxiety), with DialoFlow showing severe anxiety in multi-turn inquiries.
- Alcohol Addiction: Over half of the chatbots exhibited positive tendencies towards alcohol addiction.
- Empathy: All chatbots scored "below average" on empathy, with scores far from the average baseline.
The paper also analyzes "failed responses" (Table 3), finding that irrelevant responses were the most common type, followed by responses lacking meaningful information.
Limitations and Future Work
- Limitations:
- The assessment relies on manual annotation for response-option alignment, which can be time-consuming and subject to annotator bias.
- The default value used for failed responses might influence the final scores, though the authors note it didn't change the overall conclusion of severe issues.
- The current framework focuses on specific dimensions; it could be extended to other mental health aspects.
- Future Work:
- The authors hope to attract more research attention to this problem and build mentally healthier chatbots.
- Their framework is designed to be scalable to new mental health dimensions.
- They plan to publicly release their framework for further research.
Summary updated