TL;DR
The inaugural AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2013) marks the formalization of a new discipline where human intellect is integrated programmatically into computational systems. By transitioning from a series of workshops to a standalone conference, the organizers aim to create a rigorous, interdisciplinary home for research that blends artificial intelligence with human perception, reasoning, and coordination.
What Problem Does This Paper Address?
While "crowdsourcing" and "human computation" have existed as buzzwords and workshop topics for years, the field lacked a dedicated, high-stakes academic venue to anchor its diverse community. This preface outlines the challenge of unifying researchers from disparate fields—including Artificial Intelligence (AI), Human-Computer Interaction (HCI), economics, and cognitive psychology—under a single rigorous framework.
The core challenge addressed is the engineering of systems that can effectively leverage the "complementarities of human and machine intellect." The authors argue that human computation is not just a tool for data labeling, but a fundamental component of larger, complex systems that require human reasoning or physical coordination to function.
Methods
As this document is a conference preface rather than a traditional research paper, the "methods" described pertain to the organizational rigor and community-building strategies used to launch the field:
- Rigorous Peer Review: The conference implemented a "labor-intensive" in-person program committee meeting at Microsoft Research to ensure high-quality selections and a shared understanding of the new field’s standards.
- Multi-Track Structure: To balance high-quality archival research with the need for rapid idea exchange in a fast-evolving field, the organizers created a dual structure: a highly selective full-paper track and a non-archival "Works-in-Progress" track.
- Hands-on Incubation: The inclusion of "CrowdCamp"—a two-day hackathon-style event—was designed to move the field from theoretical discussion to concrete prototyping.
Datasets
Not discussed in the paper. (As a preface, it focuses on the conference program rather than specific experimental data).
Key Insights and Ideas
- Programmatic Access to Intellect: The authors define HCOMP as systems that rely on "programmatic access to human intellect." This frames humans not just as users, but as functional "components" or "subroutines" within a computational architecture.
- Interdisciplinary Synergy: The "aha moment" here is the explicit link between HCOMP and AI. The authors argue that the links to AI are as strong as those to HCI or economics, suggesting that human computation is a critical path forward for AI research, particularly in areas where machine intellect currently falls short.
- The "Rhythm" of Research: The transition from workshops to an annual conference is presented as a necessary step to establish a "rhythm of scholarly research," providing the stability needed for long-term engineering and theoretical breakthroughs.
Key Results
The preface provides quantitative metrics regarding the conference's debut:
- Selectivity: The conference received 84 full-paper submissions and maintained a competitive acceptance rate of just below 30%.
- Community Engagement: Beyond full papers, the conference accepted 51 Works-in-Progress, indicating a high level of active, early-stage research in the community.
- Breadth of Content: The program successfully integrated three specialized workshops (Crowdsourcing at Scale; Human and Machine Learning in Games; and Scaling Speech, Language Understanding and Dialogue) and a tutorial on Incentives.
Limitations and Future Work
The authors acknowledge that the discipline is "new and evolving," implying that the boundaries and core principles of HCOMP are still being defined. The primary goal for the future is to ensure the HCOMP meeting becomes a permanent fixture in the academic calendar, fostering a "spectrum of disciplines" to collaborate on the engineering of hybrid human-machine systems. The organizers express a clear intent for these early "works-in-progress" to inform and stimulate future research directions that will eventually mature into full conference contributions.
Summary updated