The CALO Meeting Assistant

L. Lynn Voss, Patrick EhlenNAACL 2007

TL;DR

The CALO Meeting Assistant is an intelligent system designed to automatically capture, interpret, and organize information from meetings. It goes beyond simple transcription by integrating speech, gestures, and note-taking to create a rich, browsable record, aiming to understand meeting content within the broader context of users' professional activities.

What Problem Does This Paper Address?

Meetings are a cornerstone of collaborative work, yet effectively capturing and recalling the crucial information discussed can be a significant challenge. Traditional methods rely on manual note-taking, which can be incomplete, inconsistent, and difficult to search. While previous research has focused on automated transcription or instrumented meeting rooms, this paper addresses the need for a more comprehensive and context-aware meeting assistant. The goal is to create a system that not only records what happens in a meeting but also understands its significance within the larger workflow and professional goals of the participants, thereby enhancing collaboration and knowledge management.

Methods

The CALO Meeting Assistant employs a multimodal approach to capture and interpret meeting data. Key techniques include:

  • Speech Processing: Automatic transcription of multi-speaker conversations, with adaptation to background noise and speaker-specific language models. It also leverages prosodic cues for deeper understanding.
  • Visual Recognition: Detection of faces, gaze direction, gestures, and activities. This is enhanced through lightly-supervised and unsupervised cross-training.
  • Discourse Understanding: Recognition of dialog moves, topic segmentation, and identification of action items. Discussions can be threaded across various documents and emails.
  • Multimodal Reinforcement: Integration of speech, pen input, and text to facilitate natural communication and data capture.
  • Meeting Activity Recognition: Combining speech and note-taking activities to identify meeting phases and track agenda items and document usage.

The system is designed to be unobtrusive, leveraging cross-training, unsupervised learning, and lightweight supervision from normal user interactions (like reviewing notes). Data is processed centrally and redistributed, ensuring seamless integration with other CALO desktop functionalities.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Context is King: The most significant insight is the ambition to move beyond simply recording meeting events to understanding their meaning within the user's broader professional context. This is achieved by integrating meeting data with a larger system of agents (the CALO program).
  • Multimodal Fusion for Richness: Combining speech, gestures, and note-taking provides a much richer and more robust understanding of meeting dynamics than any single modality alone.
  • Leveraging User Interaction for Learning: The system aims to learn and improve by observing how users interact with the captured data (e.g., editing notes, adding action items), making the learning process semi-supervised and more natural.
  • Cognitive Systems for Adaptability: The CALO system's foundation as a cognitive system allows it to maintain explicit models of its capabilities and adapt to unexpected situations, a key advantage over traditional, hand-coded software.

Key Results

The paper describes a demonstration of the CALO Meeting Assistant's capabilities, including:

  • Capturing speech, pen input, and other meeting data from a standard laptop.
  • Producing an automated transcript.
  • Segmenting the meeting by topic.
  • Performing shallow discourse understanding to identify probable action items from a pre-recorded meeting.
  • A "Meeting Rapporteur" feature that provides summaries and allows participants to review and organize meeting content.

The paper emphasizes the potential for real-time capabilities, allowing users to interact with the assistant during ongoing meetings.

Limitations and Future Work

  • Real-time Interaction: While the paper discusses the potential for real-time capabilities, the current demonstration focuses on a pre-recorded meeting. Developing seamless, real-time interaction during live meetings is a key area for future work.
  • Integration with Larger CALO System: The full potential of the meeting assistant is realized when integrated with the broader CALO program, which provides the contextual understanding. The paper focuses on the meeting assistant's role within this larger framework.
  • Unobtrusive Learning: While the system aims for unobtrusive learning, further research is likely needed to optimize the balance between capturing sufficient data for learning and minimizing user burden.

Summary updated

BibTeX
@inproceedings{voss2007calo,
  author = {L. Lynn Voss and Patrick Ehlen},
  title = {{The CALO Meeting Assistant}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
}
Figure