Benchmark Data Repositories: Lessons and Recommendations

Rachel Longjohn, Markelle Kelly, Padhraic Smyth, Sameer SinghZenodo 2022 · DOI · OpenAlex

Abstract

Our dataset "repository_survey" summarizes a comprehensive survey of over 150 data repositories, characterizing their metadata documentation and standardization, data curation and validation, and tracking of dataset use in the literature. In addition, "survey_model_evaluation" includes our findings on model evaluation for five benchmark repositories. Column descriptions and further details can be found in "README.pdf." The data are associated with our paper "Benchmark Data Repositories: Lessons and Recommendations."
BibTeX
@inproceedings{longjohn2022benchmark,
  author = {Rachel Longjohn and Markelle Kelly and Padhraic Smyth and Sameer Singh},
  title = {{Benchmark Data Repositories: Lessons and Recommendations}},
  booktitle = {Zenodo},
  year = {2022},
  doi = {10.5281/zenodo.8397028},
}

Loading related content...

Figure