The ContactFinder Agent: Answering Bulletin Board Questions with Referrals

AAAI 1996

TL;DR

The ContactFinder agent tackles the challenge of finding expert answers on busy electronic bulletin boards by proactively identifying and referring users to human experts, rather than trying to directly answer questions itself. This approach leverages heuristics to extract topic areas and contact information from messages, proving effective in connecting users with the right people for their queries.

What Problem Does This Paper Address?

The explosion of information on the internet, particularly in electronic bulletin board systems, has created a significant challenge: finding high-quality, timely answers to specific questions. As bulletin boards grow, it becomes less likely that true experts will regularly monitor every board. Furthermore, the sheer volume of messages leads to frequent deletion or migration, making it difficult to search for past solutions. This creates a frustrating experience for users seeking quick help. The paper addresses this by proposing an intelligent agent, ContactFinder, that acts as a referral service, connecting users with human experts rather than attempting to directly answer questions.

Methods

ContactFinder operates in two phases:

  1. Extracting Key Contacts: The agent scans incoming messages for indications of expertise. It uses heuristics to identify potential "key contacts" and their associated technical areas. This is not a general document understanding approach but a focused information extraction process.
  2. Answering Questions with Referrals: ContactFinder then monitors discussions for questions. It extracts the topic areas of these questions and searches its database for a previously identified contact relevant to that topic. If a suitable contact is found, ContactFinder posts a referral message to the bulletin board, including the contact's name, information, and a reference to the document that served as the basis for the referral.

The core of ContactFinder's approach lies in its knowledge-free heuristics for extracting semantically significant phrases. These heuristics focus on syntactic structure rather than domain-specific knowledge. Examples include:

  • Identifying fully capitalized words as potential acronyms or technical names.
  • Extracting phrases that are italicized, underlined, or capitalized for emphasis.
  • Recognizing lists, section headings, and other structural cues.

Datasets

The paper describes ContactFinder's operation on a large-scale internal bulletin board for discussion of technical issues. The evaluation was based on 2893 total documents processed from this bulletin board. No new benchmarks were introduced; the system was evaluated on existing operational data.

Key Insights and Ideas

  • Referrals over Direct Answers: The most significant insight is the effectiveness of an agent acting as a referral service. By connecting users to human experts, ContactFinder bypasses the limitations of an agent's direct knowledge and leverages the vastness of human expertise.
  • Proactive vs. Reactive: ContactFinder's proactive approach of reading and responding to messages, rather than waiting for explicit user queries, allows it to identify potential experts and questions before they become overwhelming.
  • Heuristics for Unstructured Text: The reliance on knowledge-free heuristics for extracting topic areas and contact information from unstructured text is a key innovation. This makes the system broadly applicable without requiring extensive domain-specific training.
  • Information Density and Connectivity: By posting referrals back to the bulletin boards, ContactFinder not only helps the immediate questioner but also increases the overall information density and connectivity of the system for all users.
  • Facilitating Human Expertise Transfer: The agent's function aligns well with current work styles by facilitating the transfer of expertise between people, making the system more practical and easier to adopt.

Key Results

  • ContactFinder processed 2893 documents and extracted 762 key contacts.
  • The system identified 611 questions and found 83 potential referrals.
  • This resulted in a 13.6% success rate for referrals.
  • After excluding referrals related to the complex SAP system, the success rate rose to 78% (39 approved out of 80).
  • Anecdotal feedback suggests that ContactFinder increases the amount of on-line information by encouraging experts to post responses they might not have otherwise seen.

Limitations and Future Work

  • Knowledge-Free Limitations: The knowledge-free nature of the heuristics, while a strength for applicability, becomes a limitation when dealing with highly specific domains like SAP, where acronyms and sub-system names require contextual knowledge. Future work will explore incorporating domain-specific knowledge.
  • Extracting from Diverse Documents: A key open question is how to effectively mine a wide variety of document types for contact indications.
  • Processing Plain Text: The paper acknowledges the challenge of effectively processing plain formatted text to extract contacts, questions, and subject areas.
  • Scalability and Effectiveness: Further research is needed to determine the range and effectiveness of intelligent agents built on these core document processing techniques. The authors are also developing agents for on-line service interaction and information extraction from online documents.

Summary updated

BibTeX
@inproceedings{1996contactfinder,
  title = {{The ContactFinder Agent: Answering Bulletin Board Questions with Referrals}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1996},
}

Loading related content...

Figure