TL;DR
Airbnb researchers developed a "Smart Instant Book Filter" that intelligently applies search filters to prioritize listings that can be booked immediately. By framing the problem as a dual-objective optimization—maximizing instant bookings while ensuring total booking rates remain stable—they successfully increased instant book adoption by 8.2% without hurting overall marketplace performance.
What Problem Does This Paper Address?
In online hospitality marketplaces like Airbnb, there are generally two ways to book: Request-to-Book (RTB), which requires manual host approval, and Instant Book (IB), which confirms the reservation immediately. RTB often introduces friction, such as response delays or rejections, which can frustrate guests.
While the obvious solution is to promote IB listings, doing so blindly can backfire. If a search engine hides high-quality RTB listings in favor of lower-quality IB ones, the overall booking rate (the "social welfare" of the platform) drops. The challenge is determining exactly when the available IB supply is strong enough to justify filtering out RTB options without nudging guests away from the "perfect" home that just happens to require a request.
Methods
The authors treat the marketplace as a bipartite graph where guests and listings are nodes, and bookings are edges. Because they cannot know for certain if a specific filter will result in a booking at the time of search, they use Discounted Cumulative Gain (DCG) as a functional proxy for booking probability.
The Model
The system uses a deterministic decision tree, $DT(\omega)$, which takes an information vector ($\omega$) as input. This vector includes real-time data like the number of available listings, the number of IB listings, and the current DCG of the search results. The tree outputs a boolean: should the smart filter be triggered?
Optimization Strategy
To find the optimal thresholds for this decision tree, the team used a random walk parameter exploration with restarts.
- Initialization: They start with human-expert-picked parameters.
- Mutation: They randomly adjust thresholds in the branching value vector.
- Simulation: Each mutation is tested using an in-house "Side-by-Side" (SxS) ranker on 10,000 search iterations.
- Selection: They keep the parameters that maximize the Instant Book DCG ($U_I$) while keeping the change in total Booking DCG ($\Delta U_B$) non-negative.
Datasets
The study relies on internal Airbnb production data.
- Simulation Data: 10,000 search iterations per episode step were used to evaluate parameter mutations offline.
- A/B Testing: The final model was deployed globally to millions of users to validate the offline findings against real-world behavior.
Key Insights and Ideas
- DCG as a Reliable Proxy: A major contribution is the validation that DCG—typically a search relevance metric—serves as a monotonic proxy for real-world booking behavior. Changes in averaged DCG correlated directly with percentage changes in actual bookings.
- Interpretation over Complexity: The authors chose a deterministic decision tree specifically for its high interpretability, which is crucial for debugging and understanding marketplace dynamics in a production environment.
- The "Nudge" Effect: The paper reveals a "second-order effect": by applying the filter on the guest side, they saw a +0.96% increase in hosts adopting Instant Book. This suggests that guest-side UI changes can effectively influence host-side behavior, blurring the lines of traditional two-sided market interventions.
Key Results
The automated optimization significantly outperformed manual tuning. While human-expert parameters often increased instant bookings, they almost always resulted in a slight decrease in total bookings (ranging from -0.17% to -1.60%).
In contrast, the Learned Parameter Set achieved:
- +8.20% gain in Instant Bookings.
- 0.00% change (neutrality) in overall bookings.
The system is now deployed in production, serving millions of users daily and contributing to a total increase in the platform's instant book percentage by more than 5%.
Limitations and Future Work
The current model relies on a deterministic tree with fixed parameters learned through simulation. The authors acknowledge that this might not capture rapidly evolving market conditions.
Future directions include: * Integrating the system with an online scoring model to capture more dynamic, real-time conditions. * Leveraging automated machine learning (AutoML) infrastructure to refine the parameter optimization process. * Exploring more complex model architectures beyond simple decision trees while maintaining the necessary performance for high-traffic search backends.
Summary updated