Abstract
Traditional "form-filling" search tools often fail to help users find their ideal choices because users frequently lack well-defined preferences at the start of a search. This paper evaluates an alternative "example-critiquing" approach, which allows users to iteratively refine their preferences by reacting to specific results. By incorporating "look-ahead" suggestions—options that may not be currently optimal but could become favorites if hidden preferences are revealed—decision accuracy significantly improves from 25% to 70%.
Introduction
When searching for complex items like apartments or flights, most web interfaces rely on a static "form-filling" paradigm. Users enter criteria, and the system returns matches. However, behavioral research suggests that human preferences are not pre-retrieved from memory but are instead constructed during the decision-making process. Forcing users to state all preferences upfront often leads to inaccurate data and sub-optimal outcomes due to the "anchoring effect."
Problem Statement
The fundamental flaw in traditional search tools is the assumption that users have a complete and accurate preference model before seeing any examples. In reality, users learn what they want by observing what is available. A static form prevents this discovery process, often trapping users in a "local optimum" where they settle for a choice that isn't actually their favorite among all possible options.
Methodology: Example-Critiquing and Look-ahead Suggestions
To address these limitations, the authors propose an iterative Example-Critiquing framework. Instead of a one-time form, the system engages in a dialogue: it shows examples, and the user provides feedback (e.g., "I like this, but I want a shorter commute").
The core technical innovation is the Model-Based Suggestion Strategy, governed by the "look-ahead principle." Rather than just displaying the best matches for a user's current (and likely incomplete) profile, the system identifies "suggestions" that:
- Are currently sub-optimal under stated preferences.
- Have a high probability of becoming the top choice if the user reveals a new preference (e.g., a specific airline).
This strategy uses Pareto-optimality to calculate the likelihood that a "dominated" option could become the favorite if a hidden preference is revealed, effectively "nudging" users to explore the search space more thoroughly.
Evaluation and Results
The researchers compared four versions of search tools using a database of 150 student apartments and simulated flight data. They measured Decision Accuracy (the "switch rate"): the percentage of users who, after using the tool, still preferred their choice after manually reviewing every single available option.
The results demonstrate a clear hierarchy of effectiveness:
- Traditional Form-Filling: Only 25% accuracy.
- Iterated Form-Filling: 35% accuracy.
- Basic Example-Critiquing: 45% accuracy.
- Example-Critiquing with Suggestions: 70% accuracy.
While the example-critiquing approach with suggestions took longer (approximately 7.5 minutes vs. 3 minutes for form-filling), the resulting decisions were nearly three times more likely to reflect the user's true preference.
Conclusion
The study confirms that preferences are constructed through interaction. While traditional forms are faster, they are significantly less effective for complex decision-making. By shifting to an example-critiquing model with proactive suggestions, search tools can help users escape local optima and find choices they truly value. Future work will look into handling more complex, dependent preference models and providing better support for direct trade-offs between competing options.
Summary updated