Simulating User Interaction and Search Behaviour in Digital Libraries

Saber Zerhoudi, Michael Granitzer, Christin Seifert, Jörg SchlöttererIRCDL 2022 · OpenReview

Introduction

In the landscape of web search, major commercial engines leverage vast quantities of log data to refine personalization and optimize user experience. However, digital libraries—specialized academic search engines—often face a "cold start" problem due to lower interaction volumes. Furthermore, the collection and storage of detailed user search paths present significant privacy and confidentiality risks.

To address these challenges, researchers have explored the development of user simulators capable of generating high-fidelity synthetic log data. This paper presents a suite of Markov model variants designed to simulate realistic user search sessions, enabling scalable A/B testing and system optimization without compromising user privacy.

Modeling Search Behavior: Markov Model Variants

The core of the research involves comparing five distinct Markov model architectures to determine which best captures the nuances of human search behavior:

  1. Basic Markov Model: A first-order baseline where the probability of the next action depends solely on the current state.
  2. Conditional Markov Model: This variant incorporates the "query index" (e.g., whether the user is on their first or third query), allowing the model to adapt as a session progresses.
  3. Contextual Markov Model: Users are segmented into "exploratory" (browsing) or "known-item" (targeted search) groups based on behavioral indicators like scroll depth and task completion time. Separate models are then trained for each persona.
  4. Time-aware Markov Model: This model integrates "dwell time" by fitting transition durations to a Gamma distribution, simulating how long a user lingers on specific pages.
  5. Query-change Markov Model: Utilizing a syntactic taxonomy and Longest Common Subsequence (LCS) analysis, this model tracks how users refine their queries (e.g., adding or removing terms) rather than treating every search as a generic event.

Experimental Setup and Dataset

The models were trained and evaluated using the Sowiport User Search Session (SUSS) dataset, a comprehensive log from a social science digital library. The dataset comprises over 480,000 search sessions and approximately 8 million log entries, encompassing 58 distinct user actions such as facet selection, record viewing, and citation exporting.

Results and Discussion

The efficacy of the simulated logs was measured through statistical similarity tests (Kolmogorov-Smirnov) and a "Turing test" for logs, where a classifier attempted to distinguish between real and synthetic sessions.

  • Statistical Fidelity: The Kolmogorov-Smirnov test confirmed that all models successfully produced data following the same distribution as the real-world logs.
  • Model Performance: The Conditional Markov model demonstrated the highest fidelity, achieving a 22.4% relative improvement in F-measure over the baseline. The Query-change model followed closely with an 18.7% improvement, highlighting the importance of modeling query evolution.
  • The Role of Intent: The success of the Contextual model underscored that user intent—whether exploratory or targeted—is a critical factor in behavioral modeling. Exploratory searchers behave fundamentally differently from those seeking a specific item.
  • Temporal Dynamics: Interestingly, the Time-aware model provided only marginal gains (approx. 1-2%), suggesting that the sequence of actions is a more potent indicator of behavior than the time spent on each action in this specific search context.

Conclusion

This research demonstrates that Markov models can effectively simulate complex user interactions in digital libraries. By generating realistic synthetic data, these models provide a "privacy-by-design" framework for system testing, allowing developers to understand the impact of UI changes without risking sensitive user history. Future work will look toward semantic analysis of query changes and unsupervised clustering to discover more granular user personas, further bridging the gap between simulated and real-world search behavior.

Summary updated

BibTeX
@inproceedings{zerhoudi2022simulating,
  author = {Saber Zerhoudi and Michael Granitzer and Christin Seifert and Jörg Schlötterer},
  title = {{Simulating User Interaction and Search Behaviour in Digital Libraries}},
  booktitle = {IRCDL},
  year = {2022},
  url = {https://openreview.net/forum?id=HsgYSTmakZe},
}
Figure