No article text was recorded for this version.
offline-online evaluation gap (offline-online performance gap)
Version 1 of 166 papers74 conceptsfrozen 2026-10-04
Papers in this version
- Blog: CLEF NewsREEL 2016: Comparing Multi-dimensional Offline and Online Evaluation of News Recommender Systems
- Blog: Offline Evaluation of Reward-Optimizing Recommender Systems: The Case of Simulation
- Blog: A Survey of Real-World Recommender Systems: Challenges, Constraints, and Industrial Perspectives
- Blog: Debiasing Few-Shot Recommendation in Mobile Games
- Blog: Adversarial Counterfactual Learning and Evaluation for Recommender System
- Blog: On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n Recommendation
- Blog: Benchmarking News Recommendations: The CLEF NewsREEL Use Case
- Blog: Wide & Deep Learning for Recommender Systems
- Blog: Modeling Online Behavior in Recommender Systems: The Importance of Temporal Context
- Blog: On the Choice of Data for Efficient Training and Validation of End-to-End Driving Models
- Blog: Multi-word Term Embeddings Improve Lexical Product Retrieval
- Blog: Alpenglow: Open Source Recommender Framework with Time-aware Learning and Evaluation
- Blog: Validation Set Evaluation can be Wrong: An Evaluator-Generator Approach for Maximizing Online Performance of Ranking in E-commerce
- Blog: SimUSER: Simulating User Behavior with Large Language Models for Recommender System Evaluation
- Blog: Insights on Social Recommender Systems
- Blog: Parallel Ranking of Ads and Creatives in Real-Time Advertising Systems
- Blog: Causal Meta-Mediation Analysis: Inferring Dose-Response Function From Summary Statistics of Many Randomized Experiments
- Blog: Online Ranking Prediction in Non-stationary Environments
- Blog: Report on the Evaluation-as-a-Service (EaaS) Expert Workshop
- Blog: Bridging Offline and Online Experimentation: Constraint Active Search for Deployed Performance Optimization
- Blog: Overview of NewsREEL’16: Multi-dimensional Evaluation of Real-Time Stream-Recommendation Algorithms
- Blog: RvS: What is Essential for Offline RL via Supervised Learning?
- Blog: Model Selection for Production System via Automated Online Experiments
- Blog: On (Normalised) Discounted Cumulative Gain as an Offline Evaluation Metric for Top-n Recommendation
- Blog: Importance Sampling to Identify Empirically Valid Policies and their Critical Decisions
- Blog: Information Retrieval and User-Centric Recommender System Evaluation
- Blog: Deep Automated Mechanism Design for Integrating Ad Auction and Allocation in Feed
- Blog: PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation
- Blog: EvalRS: a rounded evaluation of recommender systems
- Blog: Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications
- Blog: Learning-To-Ensemble by Contextual Rank Aggregation in E-Commerce
- Blog: Improving Music Recommendation Using Distributed Representation
- Blog: Evaluating language understanding accuracy with respect to objective outcomes in a dialogue system
- Blog: DCal-Rec: A Spatio-Temporal Distribution Calibration Framework for Next-POI Recommendation
- Blog: On Generative Agents in Recommendation
- Blog: MobileBench-OL: A Comprehensive Chinese Benchmark for Evaluating Mobile GUI Agents in Real-World Environment
- Blog: Using a Critic to Promote Less Popular Candidates in a People-to-People Recommender System
- Blog: RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising
- Blog: Imitate TheWorld: A Search Engine Simulation Platform
- Blog: EvalRS 2023: Well-Rounded Recommender Systems for Real-World Deployments
- Blog: User-in-the-loop Evaluation of Multimodal LLMs for Activity Assistance
- Blog: Next-basket Recommendation Constrained by Total Cost
- Blog: IterQR: An Iterative Framework for LLM-based Query Rewrite in e-Commercial Search System
- Blog: Augmenting Limited and Biased RCTs through Pseudo-Sample Matching-Based Observational Data Fusion Method
- Blog: Beyond Pairwise Learning-To-Rank At Airbnb
- Blog: Pantheon: Personalized Multi-objective Ensemble Sort via Iterative Pareto Policy Optimization
- Blog: Measuring Item Global Residual Value for Fair Recommendation
- Blog: RankGraph: Unified Heterogeneous Graph Learning for Cross-Domain Recommendation
- Blog: Sustainable Online Reinforcement Learning for Auto-bidding
- Blog: On Gradient Boosted Decision Trees and Neural Rankers: A Case-Study on Short-Video Recommendations at ShareChat
- Blog: Looking into User’s Long-term Interests through the Lens of Conservative Evidential Learning
- Blog: Idomaar: A Framework for Multi-dimensional Benchmarking of Recommender Algorithms
- Blog: Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings
- Blog: A Qualitative Evaluation of User Preference for Link-based vs. Text-based Recommendations of Wikipedia Articles
- Blog: Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
- Blog: Self-Supervised Reinforcement Learning for Recommender Systems
- Blog: Personalized news recommendation with context trees
- Blog: Online Models for Content Optimization
- Blog: Towards Generalizing Sensorimotor Control Across Weather Conditions
- Blog: What Are We Optimizing For? A Human-centric Evaluation of Deep Learning-based Movie Recommenders
Showing 60 of 66 papers.
History & citation
Current · Permanent link
- This direction split from Closing the Offline Online Gap in Recommender Systems (77% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_offline_online_evaluation_gap_offline_on_77586a0f,
title = {offline-online evaluation gap (offline-online performance gap)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/offline-online-evaluation-gap-offline-online-performance-gap/rdr_77586a0f062640fc8f3ed84c34567ccb},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_offline_online_evaluation_gap_offline_on_77586a0f_v1,
title = {offline-online evaluation gap (offline-online performance gap)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/offline-online-evaluation-gap-offline-online-performance-gap/rdr_77586a0f062640fc8f3ed84c34567ccb/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}