Comparing IPL2 and Yahoo! Answers: A Case Study of Digital Reference and Community Based Question Answering

Dan Wu, Daqing HeiConference 2014 · DOI

Abstract

In this paper, we took IPL2 and Yahoo! Answers as the two samples for our case study of digital references and community-based Q&A sites. We examined the services of the two systems based on 200 real questions raised in IPL2 and their similar questions found in Yahoo! Answers. Question type, topic classification, answer type, and answer time delay were compared between the similar questions in these two platforms. The result analysis showed that the two systems classify their questions differently, and the types of the questions asked are different too. It took much longer time to obtain answers from IPL2, whereas different types of questions in Yahoo! Answers generated dramatically different response time. However, some differences also demonstrated that there is need to consider integrating certain ideas in the two systems.

Introduction

While modern search engines excel at simple queries, complex information needs often require human intervention. This has fostered two distinct online ecosystems: Digital Reference Services (DRS), staffed by library professionals, and Community-Based Question Answering (CQA), where a crowd of peers provides responses. This study addresses the gap in understanding how these services differ when faced with identical information needs, aiming to identify potential for integrating the high-quality authority of librarians with the rapid response times of the crowd.

Methodology and Data

The researchers employed a "truly occurred questions" approach. They sampled 200 real questions from the June 2011 transaction logs of IPL2, a prominent digital reference site. Using these as a baseline, they developed search queries to find similar questions on Yahoo! Answers. After manual filtering for high similarity, a dataset of 157 question pairs was established. Each pair was coded across five dimensions: question type, subject classification, asker intention (objective vs. subjective), answer type (direct vs. reference-based), and time delay.

Analysis of Question and Answer Types

The study revealed a clear distinction in user intent between the two platforms. IPL2 users tended to ask more "Exploratory" questions (25.5%), which are complex and difficult to resolve. Conversely, Yahoo! Answers users sought "Informational" or "Navigational" content—essentially looking for "ready-made" information.

Distribution of question types in IPL2 and Yahoo! Answers.

Subjectivity also played a major role. Approximately 34.4% of Yahoo! Answers questions sought personal recommendations or opinions, compared to only 1.5% in IPL2. This suggests users view librarians as sources of objective facts and the community as a source of subjective advice. Furthermore, librarians almost exclusively provided "Related References/Links" (98/200 cases) to establish authenticity, while Yahoo! Answers users preferred "Direct Answers" (114/157 cases), often omitting sources.

Comparative Performance and Response Times

The most significant finding was the "speed gap." The average time delay for a first answer in IPL2 was roughly 8 days (35,236 minutes), whereas Yahoo! Answers averaged only 8 hours (495 minutes).

Average time delay for answers by question type across both platforms.

As shown in the data, response variance in Yahoo! Answers was highly dependent on question type; "Definition" questions were answered in under 2 hours, while "Navigational" questions took about 12 hours. In contrast, IPL2 delays remained consistently long regardless of the question type. This difference is partly attributed to IPL2's quality control mechanisms, where 83 of the 200 questions involved professional oversight or "mutual help" between library students and experienced librarians—a feature absent in the CQA model.

Conclusion and Future Work

This preliminary study highlights the trade-offs between professional authority and community speed. Future research should explore hybrid tools that incorporate the rapid-response mechanisms of CQA into digital reference services, while implementing the professional supervision of library science into community sites to enhance answer quality. Mapping the disparate classification systems of these platforms remains a key challenge for such integration.

Summary updated

Abstract

In this paper, we took IPL2 and Yahoo! Answers as the two samples for our case study of digital references and community-based Q&A sites. We examined the services of the two systems based on 200 real questions raised in IPL2 and their similar questions found in Yahoo! Answers. Question type, topic classification, answer type, and answer time delay were compared between the similar questions in these two platforms. The result analysis showed that the two systems classify their questions differently, and the types of the questions asked are different too. It took much longer time to obtain answers from IPL2, whereas different types of questions in Yahoo! Answers generated dramatically different response time. However, some differences also demonstrated that there is need to consider integrating certain ideas in the two systems.
BibTeX
@inproceedings{wu2014comparing,
  author = {Dan Wu and Daqing He},
  title = {{Comparing IPL2 and Yahoo! Answers: A Case Study of Digital Reference and Community Based Question Answering}},
  booktitle = {iConference},
  year = {2014},
  doi = {10.9776/14315},
}
Figure