RDBench: ML Benchmark for Relational Databases

Zizhao Zhang, Yi Yang, Lutong Zou, He Wen, Tao Feng, Jiaxuan YouSubmission to ICLR 2024 · Withdrawn

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Benefiting from high-quality datasets and standardized evaluation metrics, machine learning (ML) has achieved sustained progress and widespread applications. However, while applying machine learning to relational databases, the absence of a well-established benchmark remains a significant obstacle to the development of ML. To address this issue, we introduce \textit{ML Benchmark For Relational Databases} (RDBench), a benchmark that aims to promote hierarchical, robust, and reproducible ML research on relational databases. RDBench offers hierarchical datasets of varying scales, domains, and relations. It provides three types of data: tabular data, homogeneous graphs, and heterogeneous graphs. Importantly, all data formats share the same task definition, allowing for meaningful comparisons between methods across different data formats. Reported results are averaged over the same datasets and tasks (classification or regression), further enhancing the robustness of the experimental findings. In addition to dataset construction, we conduct extensive experiments to uncover performance differences between models. To better present our proposed RDBench, we offer a user-friendly API that provides standardized formats for three types of data.
BibTeX
@misc{zhang2023rdbench,
  author = {Zizhao Zhang and Yi Yang and Lutong Zou and He Wen and Tao Feng and Jiaxuan You},
  title = {{RDBench: ML Benchmark for Relational Databases}},
  year = {2023},
  eprint = {2310.16837},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.16837},
}
Figure