Abstract
Few-shot object detection, the problem of modelling novel object detection categories with few training instances, is an emerging topic in the area of few-shot learning and object detection. Contemporary techniques can be divided into two groups: fine-tuning based and meta-learning based approaches. While meta-learning approaches aim to learn dedicated meta-models for mapping samples to novel class models, fine-tuning approaches tackle few-shot detection in a simpler manner, by adapting the detection model to novel classes through gradient based optimization. Despite their simplicity, fine-tuning based approaches typically yield competitive detection results. Based on this observation, we focus on the role of loss functions and augmentations as the force driving the fine-tuning process, and propose to tune their dynamics through meta-learning principles. The proposed training scheme, therefore, allows learning inductive biases that can boost few-shot detection, while keeping the advantages of fine-tuning based approaches. In addition, the proposed approach yields interpretable loss functions, as opposed to highly parametric and complex few-shot meta-models. The experimental results highlight the merits of the proposed scheme, with significant improvements over the strong fine-tuning based few-shot detection baselines on benchmark Pascal VOC and MS-COCO datasets, in terms of both standard and generalized few-shot performance metrics.
Introduction
Few-Shot Object Detection (FSOD) is the challenge of teaching a model to recognize new objects using only a handful of examples (e.g., 1 to 30 images). Current approaches usually fall into two camps: simple fine-tuning, which is effective but often lacks the "inductive bias" needed to learn efficiently from very little data, and meta-learning, which uses complex systems to "learn how to learn" but is prone to overfitting and difficult to interpret.
This paper introduces Meta-Tuning, a framework that bridges this gap. Instead of building a complex, "black-box" meta-model, the authors propose using Reinforcement Learning to optimize the fine-tuning process itself. By automatically discovering the best loss function dynamics and data augmentation magnitudes, the method achieves state-of-the-art performance while maintaining the simplicity and interpretability of standard fine-tuning.
Methods: The Meta-Tuning Framework
The authors propose a framework that uses a Reinforcement Learning (RL) agent (specifically the REINFORCE algorithm) to search for optimal training hyperparameters.
1. Meta-Tuning Loss Functions
The core innovation is making the classification loss flexible. They introduce:
- Dynamic Temperature Scaling: Instead of a static Softmax, they use a temperature parameter that changes over time based on a learned polynomial function. This allows the model to "sharpen" its predictions as training progresses.
- Novel Class Scaling: A learned coefficient balances the scores between well-known "base" classes and the "novel" few-shot classes, preventing the model from ignoring new categories.
2. Meta-Tuning Augmentations
The RL agent also learns the optimal shared magnitude for photometric augmentations (brightness, contrast, saturation, hue). This ensures the data pipeline is perfectly tailored for the specific few-shot task at hand.
3. The Proxy Task Strategy
To train the RL agent without "cheating" on actual novel classes, the authors divide the base dataset into "proxy-base" and "proxy-novel" sets. The agent practices on these proxy tasks to find parameters that generalize well, which are then applied to the real few-shot classes.
Experimental Results
The approach was evaluated on the Pascal VOC and MS-COCO benchmarks, consistently outperforming strong baselines:
- MS-COCO (10-shot): Improved the MPSR baseline from 9.1 to 12.5 mAP.
- Pascal VOC: When combined with the state-of-the-art DeFRCN model, it achieved top results across almost all k-shot settings, hitting 61.3 mAP on 5-shot.
- Generalized FSOD: The method significantly improved "Harmonic Mean" scores, proving it helps the model learn new classes without degrading performance on original base classes.
Discussion and Insights
- Sharpening the Focus: The learned dynamic temperature curves show that the model prefers "sharper" probability distributions toward the end of fine-tuning, forcing more confident predictions for novel classes.
- Interpretability: Unlike meta-networks with millions of parameters, a temperature curve or an augmentation magnitude is human-readable. We can actually see what the meta-learner discovered.
- Transferability: Interestingly, loss parameters meta-tuned on one base model (like MPSR) improved performance when transferred to a completely different model (like DeFRCN), suggesting these "learned biases" are fundamental to the few-shot problem.
Limitations and Future Work
The paper focuses primarily on classification loss and photometric augmentations. The authors note that meta-tuning other components—like localization loss or initial model weights—remains unexplored. Additionally, while final fine-tuning is fast, the initial RL search for optimal parameters requires running many training episodes on proxy tasks, adding offline computational overhead.
Summary updated