Introduction
Traditional encyclopedias like Wikipedia are highly effective at conveying "know-what" facts through text and static images. However, they often fall short when it comes to "know-how" (skills) or "know-why" (complex processes). For instance, while a text entry can describe a Shiba Inu's origin, a short video is far more effective at demonstrating how to train one or even how to draw a stylized version of one.
With the explosion of short-video platforms like TikTok and Kuaishou, a vast amount of procedural knowledge is being generated daily. Kuaipedia addresses the challenge of structuring this "hidden" knowledge by linking over 200 million videos to 26 million entities and their specific sub-topics, or "aspects."
Methodology: Building the Multi-modal Encyclopedia
The construction of Kuaipedia involves a three-stage pipeline designed to filter, categorize, and link video content to a structured knowledge base.
1. Knowledge Video Detection
To distinguish educational content from pure entertainment, a multi-modal BERT-based classifier is employed. This model processes multiple data streams—including video captions, cover images, OCR text (on-screen text), and ASR (speech-to-text)—to identify "knowledge-dense" clips.
2. Item-Aspect Mining
A core innovation of Kuaipedia is the use of "aspects" to bridge broad entities and specific videos. Rather than using static categories, the system mines aspects from millions of user search queries. This ensures the encyclopedia reflects real-world utility. For an item like "Shiba Inu," aspects might include "feeding," "training," or "how-to-draw."
3. Multi-modal Item-Aspect Linking
The final stage links specific videos to Item-Aspect pairs using a two-phase Recall and Ranking model. The ranking module utilizes a task-specific prompt: "Is the video about [Item]’s [Aspect]?" evaluating the relevance of visual and textual features against the knowledge tree.
Data and Scale
Kuaipedia leverages data from Kuaishou, one of China's largest short-video platforms, and uses CN-Wikipedia and CN-DBpedia as baseline entity sources. The resulting benchmark is massive, containing over 26 million items, 2.5 million aspects, and 200 million knowledge videos, making it significantly larger than existing Chinese text-based knowledge bases.
Key Results and Impact
- Factuality in LLMs: Kuaipedia significantly reduces hallucinations in Large Language Models (LLMs). By providing multi-modal context (OCR/ASR text), researchers improved ChatGPT's accuracy on specific factual queries from 0% to 100% in targeted test cases.
- Enhanced AI "Imagination": Feeding Kuaipedia's video frames and ASR text into text-to-image models allows AI to visualize complex actions more accurately than text prompts alone.
- Performance Gains: The inclusion of Kuaipedia data improved F1 scores in traditional NLP tasks such as Entity Typing (from 97.43 to 98.12) and Entity Linking (from 79.04 to 81.22).
Conclusion
Kuaipedia represents a significant step forward in knowledge representation by moving beyond text to capture the procedural richness of video. By structuring "know-how" through an item-aspect framework, it provides a valuable resource for both human users and AI systems, grounding abstract concepts in visual reality.
Summary updated