This paper, recently published by researchers from the University of Washington and Microsoft Research, explores the shifting landscape of how we find and use information in the age of Generative AI (GenAI).
TL;DR
The paper introduces the concept of panmodal information interaction, moving beyond simple "search vs. chat" to a future where multiple modes of interaction are seamlessly integrated. Based on a user survey, the authors argue that while search is for "finding," chat is for "doing," and the next generation of AI systems must support the fluid transitions (crossmodal) and deep integration (transmodal) between these modes.
What Problem Does This Paper Address?
For decades, the "10 blue links" of search engines like Google and Bing were the gold standard for information retrieval. The sudden rise of GenAI agents like ChatGPT has introduced a second, powerful modality: natural language chat.
Currently, these two worlds often exist as separate silos or are "loosely coupled" (e.g., a chat tab next to a search bar). The authors identify a gap in our understanding of how users choose between these modes, why they switch from one to the other, and how systems can be designed to support a "seamless flow" rather than forcing users to pick one or the other.
Methods
The authors conducted a survey of several hundred crowdworkers (resulting in 130 high-quality, analyzed responses) to gather personal accounts of using search and chat for professional and personal tasks.
They used these insights to propose a new taxonomy for information interaction:
- Monomodal: Using only one modality (e.g., search only).
- Multimodal: Multiple modalities are available and used for a task.
- Crossmodal: Directly supporting the switch from one modality to another (e.g., starting in search and moving to chat to refine the result).
- Transmodal: An integrated experience where the task spans multiple modalities simultaneously (e.g., a search engine finding sources while a chat agent drafts a report based on them).
Datasets
The primary data source is a qualitative survey of 130 US-based, English-speaking participants. The survey included:
- Multiple-choice questions on modality preference for specific tasks.
- Open-ended text responses describing successes and failures in search and chat.
- Demographic data (60% male, 39% female, 1% other).
Key Insights and Ideas
- The "Search for Finding, Chat for Doing" Divide: Users overwhelmingly prefer traditional search for informational and planning tasks (like finding a vacation spot or event info). Conversely, chat-based modalities dominate for "understanding" and "creation" tasks (like explaining scientific concepts or writing reports).
- The One-Way Street: The survey revealed a significant "crossmodal" flow from search to chat. Users often start with a broad search and switch to chat when they need the information tailored or made interactive. Interestingly, in the survey data, almost no users switched from chat back to search for the same task.
- Mental Models are Still Forming: While users have decades of experience with search, they are still experimenting with "prompt engineering" and forming mental models of what GenAI can and cannot do.
- Search's "Trust" Advantage: Search is still the go-to for "Trustworthy and Verified Sources" and "Real-Time/Location-Based Info," whereas chat is criticized for its "Limited Creativity on Sensitive Topics" and "Difficulty Following Complex Instructions."
Key Results
- Task Preferences:
- Search: Preferred for "Planning a vacation" (61%) and "Looking for event info" (58%).
- Chat: Preferred for "Writing a work-related report" (60%) and "Explaining photosynthesis" (55%).
- Switching Behavior: In a specific analysis of 123 respondents, 15 users started in search and switched to chat to finish their task. Zero users started in chat and switched to search.
- Qualitative Summary: The authors produced a comprehensive table (Table 1) mapping the specific strengths and challenges of both modalities. For example, search's biggest challenge is "Irrelevant Results and Overwhelming Information," while chat's biggest hurdle is "Inadequate for Quick Searches/General Info."
Limitations and Future Work
- Sample Size and Diversity: The study relied on a relatively small group (130) of US-based crowdworkers. The authors acknowledge that usage patterns may differ across different cultures and languages.
- Lack of "Transmodal" Examples: Because current commercial systems are mostly "multimodal" (offering both but keeping them separate), the authors had to rely on user descriptions of "ad hoc workflows" rather than observing true transmodal system usage.
- Future Directions: The authors call for new "routing mechanisms" that can automatically suggest the best modality for a specific step in a task. They also envision expanding panmodality to include audio, vision, touch (tactition), and eye-gaze tracking.
Summary updated