TL;DR
This paper introduces MicDialogue, the first large-scale Chinese dialogue dataset specifically for maternity and infant care, addressing the lack of domain-specific data for LLMs in this crucial area. They also propose Kng-BART, a knowledge-enhanced dialogue generation model that leverages external knowledge graphs to produce more relevant and empathetic responses.
What Problem Does This Paper Address?
The rapid advancement of Large Language Models (LLMs) has revolutionized human-computer dialogue, but their performance often falters in specialized domains due to a scarcity of relevant data. Maternity and infant care, a field requiring deep knowledge in gynecology, pediatrics, and emotional well-being, is particularly underserved. Existing dialogue datasets are either too general or lack the nuanced understanding of real-time interactions, emotional complexities, and specific medical topics crucial for this domain. This gap hinders the development of effective AI assistants that can provide genuine support to expectant and new parents.
Methods
The authors tackle this problem through a multi-pronged approach:
- Dataset Curation (MicDialogue): They meticulously collected dialogues from Chinese social media platforms, specifically short videos (like Kuaishou, Douyin) and blog posts (Weibo, Zhihu). Short videos capture dynamic, real-time interactions, while blog posts offer comprehensive discussions. The data was then processed to remove noise and structured into multi-turn dialogues.
- Rigorous Annotation: A detailed annotation scheme was developed, labeling dialogues with topics (gynecological health, pediatric care, pregnancy preparation, emotional counseling), diseases, symptoms, and causes. This rich metadata is crucial for fine-grained analysis and model training.
- Knowledge-Enhanced Dialogue Generation (Kng-BART): To improve response generation, they developed a novel benchmark model. This model uses LLMs for knowledge expansion and selection, then leverages graph-based knowledge representation from multiple knowledge graphs (ConceptNet, medical, and mental health KGs). A graph attention network and an utterance-knowledge cross-attention mechanism are employed to integrate this external knowledge into the dialogue generation process.
Datasets
- MicDialogue: A newly introduced Chinese dialogue dataset for maternity and infant care. It comprises 8,813 dialogues sourced from blog posts (7,685) and short videos (1,128).
- Existing Datasets (for comparison): MedDialog-EN, MedDialog-CN, IMCS-21, ChatDoctor, RealMedDial, EmpatheticDialogues, ESConv, IEmphathy, MTFC, WDJN, Gender-Specific Dialogue.
Key Insights and Ideas
- The Value of Diverse Data Sources: Combining short videos and blog posts provides a richer, more dynamic dataset than relying on a single source. Short videos capture the immediacy of real-world conversations, while blogs offer depth and breadth.
- Metadata is King: The detailed annotation of topics, diseases, symptoms, and causes is a critical differentiator, enabling more sophisticated downstream tasks beyond simple response generation.
- External Knowledge is Essential for Specialization: Generic LLMs struggle with domain-specific nuances. Integrating knowledge graphs significantly enhances the model's ability to generate relevant and accurate responses in specialized fields like maternity and infant care.
- Emotional Nuance is Paramount: Maternity and infant care dialogues are heavily laden with emotional content. The dataset and model explicitly address this, aiming to provide empathetic support.
Key Results
- MicDialogue Dataset: Contains 8,813 dialogues with detailed annotations, making it the first of its kind for maternity and infant care.
- Kng-BART Performance: Outperformed other models (Transformer, BART, ChatGPT, CDial-GPT) across various automatic evaluation metrics (BLEU, NIST, METEOR, DIST-2) on the MicDialogue dataset.
- Human Evaluation: Kng-BART also showed superior performance in human evaluations for relevance, informativeness, and topic alignment, indicating its effectiveness in generating high-quality, domain-specific responses.
Limitations and Future Work
- Limitations: The paper acknowledges that while Kng-BART is more fluent than ChatGPT, it is slightly less so. The dataset, while comprehensive, is primarily in Chinese, limiting its direct applicability to other languages without translation.
- Future Work: The authors suggest exploring the generalization of their approach to other AI sub-fields by adapting knowledge graphs and annotation strategies. They also aim to develop highly specialized LLMs for maternity and infant care and delve deeper into mental health aspects within this domain.
Summary updated