AIR 2008: second international workshop on adaptive information retrieval

Hideo Joho, Frank Hopfgartner, Joemon M. Jose, C. J. van RijsbergenSIGIR 2009 · DOI · OpenReview

Introduction

The Second International Workshop on Adaptive Information Retrieval (AIR 2008) addressed a fundamental limitation in traditional search systems: the "one-size-fits-all" approach. While standard Information Retrieval (IR) models often treat search as a static transaction—a query followed by a list of results—real-world information seeking is a dynamic, context-dependent process. AIR 2008 brought together researchers to bridge the "context gap" and develop systems that adapt to specific user behaviors, tasks, and environments.

Theoretical Frameworks for Interaction

A central theme of the workshop was the need for a unified theoretical foundation for Interactive IR (IIR). Norbert Fuhr proposed a significant extension of the Probability Ranking Principle (PRP) to interactive settings. Rather than optimizing for a single query, this extended model accounts for the costs and evolving needs of a user throughout an entire search session. This shift treats search as a continuous dialogue where the system must balance the utility of information against the cognitive and temporal costs to the user.

Instead of attempting to construct a grand, top-down model of human-computer interaction, participants suggested a "bottom-up" approach to theory. This involves integrating individual adaptive components—such as user modeling and content adaptation—to observe their interactions and eventually derive a broader theoretical framework.

Adaptive Interfaces and Contextual Tools

Beyond theory, the workshop explored practical applications and interfaces designed to handle the complexity of user context. Key presentations included:

  • Contextual Control: Graphical interfaces that allow users to manually adjust the weight of various contextual factors, giving them agency over how the system adapts.
  • Cross-Modal Retrieval: Innovative hardware, such as camera-equipped pens, designed to bridge the gap between physical documents and digital retrieval systems.
  • Workflow Integration: Multi-level frameworks aimed at academic environments that unify adaptive services and content to streamline research workflows.

David Harper (Google) emphasized that the "typical user" is a myth. Users vary significantly by location, language, and expertise, requiring systems that can present information in diverse formats tailored to these specific attributes.

The Evaluation Bottleneck

One of the most significant challenges identified was the inadequacy of traditional evaluation metrics. The classic "Cranfield" approach, which relies on static test collections and pre-determined gold standards, cannot effectively measure an adaptive system. Because an adaptive system's behavior changes based on interaction, its "correctness" is a moving target.

The community reached a consensus on the urgent need for:

  1. Interactive Test Collections: Datasets that capture user interactions over time rather than just single-query snapshots.
  2. New Metrics: Measures that account for user effort, satisfaction, and success across a full session.

Conclusion and Future Directions

AIR 2008 served as a roadmap for the evolution of search. By identifying the three pillars of adaptive interfaces, interactive evaluation, and models of interaction, the workshop set the stage for subsequent research. The transition from simple matching algorithms to cost-sensitive, interactive models remains a primary goal for the field. Future work, as signaled by the workshop's conclusion, will focus on large-scale logging and the interpretation of user interactions to build truly data-driven adaptive systems.

Summary updated

BibTeX
@article{joho2009air,
  author = {Hideo Joho and Frank Hopfgartner and Joemon M. Jose and C. J. van Rijsbergen},
  title = {{AIR 2008: second international workshop on adaptive information retrieval}},
  journal = {SIGIR},
  year = {2009},
  doi = {10.1145/1670598.1670611},
}
Figure