Beyond the Lookup: The Challenge of Complex Search
Most modern search engines are optimized for "telepathic" lookup—situations where a user provides a specific query and expects a direct answer. However, real-world information seeking is often significantly more intricate. Whether a professional researcher is curating a dataset or a parent is investigating how to "make a phone safe for a child," the search process is rarely a single event. Instead, it is a multi-stage, exploratory journey.
The 2017 CHIIR workshop on "Supporting Complex Search Tasks" (SCST) addressed the fragmentation in our understanding of these processes. Unlike simple lookups, complex tasks involve evolving information needs, multiple sessions, and a continuous cycle of searching, browsing, and sense-making. Current Information Retrieval (IR) systems often fail here because they treat every query as an isolated, identical step, rather than supporting the user’s transition from broad exploration to focused refinement.
Synthesizing Perspectives: The SCST Workshop
The SCST workshop functioned as an interdisciplinary bridge between Human-Computer Interaction (CHI) and Information Retrieval (IR) researchers. Rather than focusing on a single algorithm, the forum synthesized diverse contributions to identify barriers to success in complex search.
The workshop featured:
- Keynote Perspectives: Discussions on bridging the gap between academic theories of work tasks and the industrial reality of complex user needs.
- Peer-Reviewed Contributions: Nine short papers exploring topics such as concept mapping, visual analytics for exploratory search, and frameworks for search UI features.
- Collaborative Breakouts: Sessions aimed at brainstorming new evaluation tracks to better capture the nuances of interactive search.
Benchmarking Complexity
A significant challenge in this field is the lack of standardized datasets that capture the temporal and contextual nature of complex tasks. The workshop highlighted several foundational benchmarks:
- Social Book Search (SBS) Lab at CLEF: Specifically the Interactive Track (2014–2016), which provides scenarios for complex, multi-faceted book-seeking.
- TREC Session and Tasks Tracks: Benchmarks designed to evaluate sequences of queries within a single task.
- CHiC (Cultural Heritage in CLEF): A resource for studying interactive retrieval within domain-specific collections.
The authors emphasized a persistent need for more realistic evaluation campaigns to ground simulated interactive search in empirical evidence.
Key Insights: Search as Learning and Contextual Design
The workshop yielded several critical insights into the future of search systems:
- The "Simple" Task Fallacy: Tasks that appear simple on the surface (e.g., "make a phone safe") are often complex due to conflicting information, hardware variations, and the absence of a single "correct" answer.
- Search as Knowledge Evolution: Research by Egusa et al. indicates that users' internal "concept maps" change significantly during a complex search. This suggests that search tools should be designed as learning environments.
- UI/UX Affordances: Designing interfaces for complex search requires a delicate balance. Systems must provide tools for exploration without overwhelming the user, maintaining transparency while determining when the system should take the initiative.
- Integrated Context: Context should not be a mere filter but an integral part of the search. An "outside-in" approach defines the work context first, followed by the task, and finally the search variables.
Advancing the Field: Frameworks and Metaphors
The workshop showcased specific advancements in supporting complex tasks. For instance, Huurdeman proposed a three-part UI feature set—personalizable, informational, and input/control—mapped to different stages of the search process. In another contribution, Ventocilla et al. demonstrated a "bottom-up" exploratory approach using a "billiard metaphor" to visualize correlations in data sets, allowing for more intuitive exploration of complex information spaces.
Future Directions in Interactive IR
While the workshop fostered a successful collaborative environment, several hurdles remain:
- The Evaluation Gap: There is currently no "gold standard" for measuring success in multi-stage search. New models are needed to combine topical relevance with contextual and user preferences.
- System Initiative: Determining the optimal moment for a system to offer recommendations—versus staying out of the user's way—remains an open question.
- Community Integration: There is a call for tighter integration between system-centered (ranking/algorithms) and user-centered (UI/UX) research.
The workshop concluded that the next phase of research must focus on developing alternative experiments and prototypes to be tested against sophisticated new benchmarks, such as the planned TREC Tasks Track.
Summary updated