HPT4Rec: AutoML-based Hyperparameter Self-Tuning Framework for Session-based Recommender Systems

Amir Reza Mohammadi, Amir-Hossein Karimi, Mahdi Bohlouli, Eva Zangerle, Günther SpechtGvDB 2023 · OpenReview

TL;DR

This paper introduces HPT4Rec, an AutoML framework designed to automate hyperparameter tuning for session-based recommender systems. By providing a modular, extensible codebase and integrating various tuning strategies, HPT4Rec aims to simplify the development and deployment of these systems, making them more practical for real-world applications.

What Problem Does This Paper Address?

Recommender systems (RSs) are crucial for improving user experience in e-commerce and other applications. While traditional RSs often rely on user profiles, session-based recommendations offer a compelling alternative, especially when user history is unavailable or privacy concerns limit data collection. However, developing and fine-tuning deep learning-based session-based recommenders (SBRs) is challenging. Existing frameworks often prioritize state-of-the-art accuracy over ease of development and reproducibility, hindering rapid experimentation and debugging. This makes it difficult to adapt RSs to dynamic real-world scenarios where data volume and user preferences constantly change. The paper addresses the need for a framework that automates hyperparameter tuning and offers industry-level compatibility for SBRs.

Methods

HPT4Rec is an AutoML-based framework built upon a modular deep session-based recommender codebase. Its core components include:

  • Modular Codebase: The framework utilizes PyTorch and Microsoft NNI, offering a flexible structure that allows for easy addition of new features and algorithms.
  • Automated Hyperparameter Tuning (HPT4Rec): This component automates the search for optimal hyperparameters for SBR models. It supports 11 popular self-tuning algorithms, enabling users to select the most suitable strategy based on their specific needs (e.g., limited computational resources, varying data volumes).
  • Extensible Data Structures and Training Service Compatibility: The framework is designed to be compatible with various training platforms, including local servers, distributed networks, and open-source platforms like Kubernetes and OpenPAI.
  • Session-based Modeling with RNNs: The underlying recommender model uses Gated Recurrent Units (GRUs), a variant of Recurrent Neural Networks (RNNs), which are well-suited for sequential data like user clickstreams. The paper details the GRU architecture and its application in the context of session-based recommendations.
  • Search Space Definition: Users define the hyperparameter search space using a JSON file, specifying the type and range of values for each parameter (e.g., uniform, choice, loguniform).

Datasets

The primary dataset used for experiments is the YOOCHOOSE e-commerce dataset from the RecSys 2015 challenge. This dataset comprises click-streams from an e-commerce site, including session IDs, item IDs, and purchase information. After preprocessing, it contains approximately 7.9 million sessions and 31.4 million clicks on over 37,000 items. The paper also mentions that the framework is designed to be compatible with other datasets like Diginetica, Xing, and Last.fm.

Key Insights and Ideas

  • The Need for Automation in SBRs: The paper strongly emphasizes that for SBRs to be practical in real-world scenarios, automated hyperparameter tuning is not just a convenience but a necessity. Dynamic changes in data and user behavior require continuous model optimization.
  • Modularity as a Key to Adaptability: A modular framework is crucial for allowing researchers and practitioners to easily experiment with new algorithms and adapt models to specific business needs without reinventing the wheel.
  • Tuning Strategy Matters: Different self-tuning algorithms (e.g., TPE, SMAC, Anneal) have varying strengths and are more suitable for different scenarios. HPT4Rec's support for multiple tuners allows users to choose the most efficient approach for their specific constraints.
  • Reproducibility and Ease of Use: The authors highlight that many advanced DL models are difficult to reproduce and deploy. HPT4Rec aims to bridge this gap by providing a well-documented, open-source framework that simplifies the entire process from experimentation to potential deployment.

Key Results

  • HPT4Rec achieved performance on par with state-of-the-art results on the benchmark RecSys 2015 dataset.
  • The framework demonstrated the importance of continuous and iterative parameter tuning, especially for real-world scenarios.
  • Experiments showed that while various tuning methods perform well, no single method universally outperforms others across all datasets and metrics, reinforcing the need for a flexible framework like HPT4Rec.
  • The paper presents a comparison of HPT4Rec's optimized recommender against several baseline models (POP, S-POP, Item-KNN, BPR-MF) and GRU4Rec configurations, showing competitive results.

Limitations and Future Work

  • Limitations: The paper acknowledges that while HPT4Rec simplifies tuning, a complete automation of the entire recommendation pipeline (including data augmentation) is still a long-term goal. The current work focuses on hyperparameter tuning.
  • Future Work: The authors plan to expand the framework by automating other aspects of the recommendation pipeline, such as automated data augmentation, which is currently a manual process in literature. They also aim to further enhance the framework's capabilities for real-world deployment and integration.

Summary updated

BibTeX
@inproceedings{mohammadi2023hpt,
  author = {Amir Reza Mohammadi and Amir-Hossein Karimi and Mahdi Bohlouli and Eva Zangerle and Günther Specht},
  title = {{HPT4Rec: AutoML-based Hyperparameter Self-Tuning Framework for Session-based Recommender Systems}},
  booktitle = {GvDB},
  year = {2023},
  url = {https://openreview.net/forum?id=CzWV6PLUEQ},
}
Figure