Cobi: Community-Informed Conference Scheduling

Juho Kim, Haoqi Zhang, Paul André, Lydia B. Chilton, Anant P. Bhardwaj, David R. Karger, Steven P. Dow, Robert C. MillerHCOMP (Works in Progress / Demos) 2013 · OpenReview

Abstract

Creating a schedule for a large multi-track conference requires considering the preferences and constraints of organizers, authors, and attendees. Traditionally, a few dedicated organizers manage the size and complexity of the schedule with limited information and coverage. Cobi presents an alternative approach to conference scheduling by engaging the entire community to take active roles in the planning process. It consists of a collection of crowdsourcing applications that elicit preferences and constraints from the community, and software that enable organizers and other community members to take informed actions based on collected information.

Introduction

Organizing a large-scale academic conference like CHI (Human-Computer Interaction) is a monumental logistical challenge. With over 2,000 submissions and hundreds of accepted papers distributed across 16 parallel tracks, the traditional "top-down" approach to scheduling is reaching its limits. Historically, a small group of chairs would manually move physical paper representations on a board, a process fraught with information bottlenecks. Organizers cannot realistically grasp the nuances of every sub-field, leading to incoherent sessions, thematic conflicts (where similar sessions overlap), and unresolved author constraints.

Cobi (Community-Informed) transforms this manual chore into a collaborative, data-driven process. By distributing the workload across the entire community—including authors and committee members—Cobi provides organizers with the granular affinity data and intelligent visual tools necessary to create cohesive, conflict-free schedules for massive multi-track events.

The Cobi Methodology

Cobi addresses scheduling complexity by breaking it down into manageable micro-tasks distributed through specialized applications.

1. Committee Clustering

Expert input is gathered from Associate Chairs (ACs) who understand the specific sub-fields. Cobi employs two primary interfaces: Partial Clustering and Cascade. In the Cascade interface, contributors first generate category names and then determine if specific papers fit those categories. This structured approach helps build a bottom-up taxonomy of the conference's content.

The Cascade interface allows committee members to categorize papers through micro-tasks, helping to define thematic clusters.

2. Author Refinement

Authors of accepted papers are uniquely motivated to ensure their work is placed in a relevant session. Cobi leverages this incentive by inviting authors to identify papers that would complement theirs in a session. To prevent information overload, Cobi uses a combination of committee-identified affinities and TF-IDF (Term Frequency-Inverse Document Frequency) analysis of titles and abstracts to present authors with a curated "shortlist" of potential matches.

3. Intelligent Scheduling Interface

The data collected from the community is integrated into a visual "cockpit" for conference chairs. This interface acts as an intelligent assistant rather than a fully automated scheduler. When an organizer moves a session, the tool uses constraint-solving logic to immediately highlight new conflicts and calculate the "cost" of the move. It can also recommend alternative time slots that minimize conflicts, highlighting them in green to guide the organizer's decision-making.

The intelligent scheduling interface provides organizers with visual feedback on conflicts and recommends optimal slots based on community-derived constraints.

Real-World Deployment: CHI 2013

The system was evaluated through a full-scale deployment for the CHI 2013 conference, involving over 2,000 submissions and 500+ accepted papers. The community response was significant: 645 authors provided 8,651 preferences and constraints, covering 87% of all accepted papers. Additionally, 64 associate chairs identified 1,722 pairs of related papers.

This wealth of data allowed organizers to identify and resolve 168 specific conflicts that likely would have been missed in a manual process. The resulting schedule demonstrated higher thematic consistency and fewer "stray" papers compared to previous years.

Key Insights

Cobi's success stems from several core principles:

  • Incentive Alignment: By tapping into authors' desire for relevant sessions and conflict-free speaking slots, Cobi ensures high-quality data without external compensation.
  • Micro-tasks: Distributing 10-second judgments across hundreds of people captures fine-grained data that no single human or algorithm could produce alone.
  • Augmented Intelligence: The system prioritizes human agency. It handles the data-heavy constraint checking, allowing organizers to focus on the strategic and aesthetic aspects of the schedule.

Limitations and Future Work

While successful, the CHI 2013 deployment highlighted areas for growth. Future iterations aim to incorporate attendee preferences to better predict session popularity and optimize room assignments. Additionally, the authors plan to transition from asynchronous tools to real-time collaborative environments for taxonomy building. As the volume of community data grows, the recommendation engines must also evolve to remain computationally efficient and user-friendly.

Summary updated

Abstract

Creating a schedule for a large multi-track conference requires considering the preferences and constraints of organizers, authors, and attendees. Traditionally, a few dedicated organizers manage the size and complexity of the schedule with limited information and coverage. Cobi presents an alternative approach to conference scheduling by engaging the entire community to take active roles in the planning process. It consists of a collection of crowdsourcing applications that elicit preferences and constraints from the community, and software that enable organizers and other community members to take informed actions based on collected information.
BibTeX
@inproceedings{kim2013cobi,
  author = {Juho Kim and Haoqi Zhang and Paul André and Lydia B. Chilton and Anant P. Bhardwaj and David R. Karger and Steven P. Dow and Robert C. Miller},
  title = {{Cobi: Community-Informed Conference Scheduling}},
  booktitle = {HCOMP (Works in Progress / Demos)},
  year = {2013},
  url = {https://openreview.net/forum?id=HolZzpQJfw},
}
Figure