Abstract
본 논문은 기존의 정보검색 시스템의 검색 결과에 비연관 문헌이 포함되는 단점과 질의응답 시스템이 가지는 자연어처리 기술의 한계를 극복하고, 사용자의 정보 요구에 실시간으로 정보를 제공하며 사용자의 참여를 적극 활용하여 웹2.0 환경으로의 변화에 적응할 수 있는 실시간 인력기반 질의응답 시스템을 제안한다. This research suggests a real-time people-powered Q&A system that overcomes the limitations of natural language handling technology in traditional Q&A systems and the irrelevance of documents in standard search engines. By actively leveraging user participation, the system provides real-time information to meet user demands in a Web 2.0 environment.
Introduction
In the late 2000s, information retrieval faced a significant challenge. While standard search engines were incredibly fast, they often produced a "document dump" of irrelevant results. Conversely, automated Question and Answering (Q&A) systems struggled with the complexities and nuances of natural language. Human-powered platforms like Naver Knowledge iN offered high-quality, nuanced answers but were fundamentally asynchronous, often leaving users waiting hours or days for a response.
This paper proposes the P2QA (People-Powered Question and Answering) system, designed to bridge this gap by combining the accuracy of human expertise with the immediacy of automated systems.
System Architecture
The P2QA system is built upon three primary modules designed to process queries and route them to the most appropriate human experts in real-time.
- Index Term Extractor: This module performs morpheme analysis on raw text to extract nouns. It employs a bi-gram approach—grouping two adjacent words—to enhance search accuracy specifically for Korean text.
- Automatic Classification Module: To determine query intent, the system utilizes a k-Nearest Neighbor (kNN) algorithm ($k=80$). Queries are mapped into a hierarchy of 17 major and 226 minor categories. TF-IDF (Term Frequency-Inverse Document Frequency) is used to weight term importance.
- Human Agent Search Module: This core component identifies the optimal human expert using a multi-factor ranking formula ($Rank_j$). The ranking considers the expert's current online status, classification match with the query, cosine similarity between the query and the expert's profile, and a reliability score based on previous user ratings.
Experimental Evaluation
The system was trained using 15,000 web documents (10,000 news articles and 5,000 Q&A pairs) and evaluated using 60 test queries across professional and daily life categories. Fifty university students participated as "Human Agents."
A key feature of the classification logic is the use of weighted matches for the top three categories. This ensures that even if the primary category isn't a perfect match, the query is still routed to a relevant expert.
Key Results
The P2QA system demonstrated significant performance improvements over contemporary benchmarks:
- Response Time: The average first-response time was just 1.53 minutes, compared to 83.9 minutes for Naver Knowledge iN.
- User Satisfaction: Users reported a Mean Opinion Score (MOS) of 3.8 out of 5.0, indicating they were "Close to Satisfied" with the quality and speed of the service.
Discussion and Future Work
The P2QA system's success hinges on treating human intelligence as a real-time queryable resource. By prioritizing "Status" in the ranking algorithm, the system transforms traditional forum-style Q&A into a live, interactive experience.
However, the system faces challenges regarding the scalability of human capital; if no experts are online for a specific niche, the real-time promise cannot be met. Additionally, new experts face a "cold start" problem due to the lack of initial reliability ratings. Future research should focus on dynamic profile updates based on actual answering behavior and the development of more objective evaluation benchmarks.
Summary updated