AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification

Xuyang Shen, Jo Plested, Sabrina Caldwell, Yiran Zhong, Tom Gedeon

Abstract

Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in target tasks. Fine-tuning is able to alleviate the challenge of insufficient training data and expensive labelling of new data. However, standard fine-tuning has limited performance in complex data distributions. To address this issue, we propose the Adaptable Multi-tuning method, which adaptively determines each data sample's fine-tuning strategy. In this framework, multiple fine-tuning settings and one policy network are defined. The policy network in Adaptable Multi-tuning can dynamically adjust to an optimal weighting to feed different samples into models that are trained using different fine-tuning strategies. Our method outperforms the standard fine-tuning approach by 1.69%, 2.79% on the datasets FGVC-Aircraft, and Describable Texture, yielding comparable performance on the datasets Stanford Cars, CIFAR-10, and Fashion-MNIST.
BibTeX
@misc{shen2022amf,
  author = {Xuyang Shen and Jo Plested and Sabrina Caldwell and Yiran Zhong and Tom Gedeon},
  title = {{AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification}},
  year = {2022},
  eprint = {2207.12944},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2207.12944},
}

Loading related content...

Figure