Independently-prepared Query-efficient Model Selection
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
The advancement of deep learning technologies is bringing new models by the day, which not only facilitates the importance of model selection but also makes it more challenging than ever. However, existing solutions for model selection either require a large amount of model operations proportional to the number of candidates when selecting models for each task, or require group preparations that jointly optimize the embedding vectors of many candidate models. As a result, the scalability of existing solutions is limited with the increasing amounts of candidates. In this work, we present a new paradigm for model selection, namely independently-prepared query-efficient model selection. The advantage of our paradigm is twofold: first, it is query-efficient, meaning that it requires only a constant amount of model operations every time it selects models for a new task; second, it is independently-prepared, meaning that any information about a candidate model that is necessary for the selection can be prepared independently requiring no interaction with others. Consequently, the new paradigm offers by definition many desirable properties for applications: updatability, decentralizability, flexibility, and certain preservation of both candidate privacy and query privacy. With the benefits uncovered, we present Standardized Embedder as a proof-of-concept solution to support the practicality of the proposed paradigm. We empirically evaluate this solution by selecting models for multiple downstream tasks, from a pool of 100 pre-trained models that cover different model architectures and various training recipes, highlighting the potential of the proposed paradigm.
BibTeX
@misc{wang2024independently,
author = {Wenxiao Wang and Weiming Zhuang and Lingjuan Lyu},
title = {{Independently-prepared Query-efficient Model Selection}},
year = {2024},
url = {https://openreview.net/forum?id=cOLzQHklmn},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}