Abstract
This paper provides an extensive analysis of the performance of the EfficientNet image classifiers with several recent training procedures, in particular one that corrects the discrepancy between train and test images. The resulting network, called FixEfficientNet, significantly outperforms the initial architecture with the same number of parameters. For instance, our FixEfficientNet-B0 trained without additional training data achieves 79.3% top-1 accuracy on ImageNet with 5.3M parameters. This is a +0.5% absolute improvement over the Noisy student EfficientNet-B0 trained with 300M unlabeled images. An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88.5% top-1 accuracy (top-5: 98.7%), which establishes the new state of the art for ImageNet with a single crop. These improvements are thoroughly evaluated with cleaner protocols than the one usually employed for Imagenet, and particular we show that our improvement remains in the experimental setting of ImageNet-v2, that is less prone to overfitting, and with ImageNet Real Labels. In both cases we also establish the new state of the art.
Introduction
Modern computer vision models, particularly the EfficientNet family, have pushed the boundaries of image classification. However, a subtle but pervasive "distribution shift" exists between training and testing phases. This paper from Facebook AI Research (FAIR) introduces FixEfficientNet, which resolves this discrepancy using a simple training "patch" called FixRes. This refinement allows models to reach new heights, including a state-of-the-art 88.5% top-1 accuracy on ImageNet.
The Resolution Discrepancy Problem
During training, standard practice involves heavy data augmentation: a random rectangular "Region of Classification" (RoC) is extracted from an image and resized to a fixed square (e.g., 224x224). During testing, however, models typically evaluate a single, centered crop.
Because the RoC at test time differs from the random RoCs seen during training, objects appear at different scales and resolutions. This "resolution discrepancy" means models are never fully optimized for the specific image characteristics encountered during evaluation. While researchers have historically mitigated this by manually tuning test resolutions, the underlying mismatch remained unaddressed.
The FixRes Method
The authors apply the FixRes (Fixing Resolution) method to EfficientNet models through a computationally efficient three-step procedure:
- Standard Training: The model is trained at a lower resolution using standard data augmentation.
- FixRes Fine-tuning: The model is fine-tuned for a few epochs at the higher target test resolution. Crucially, only the final layers (classifier and sometimes batch normalization) are updated.
- Label Smoothing: This is integrated during fine-tuning to enhance generalization.
This approach is highly efficient because the bulk of training occurs at lower resolutions, and the final stage involves only a fraction of the network's parameters.
Experimental Results
The improvements are consistent across all model sizes and benchmarks.
Performance and Efficiency
As shown in Figure 1, FixEfficientNet consistently outperforms other variants across the entire parameter range. A FixEfficientNet-B0 (5.3M parameters) achieves 79.3% accuracy, surpassing the much larger "Noisy Student" EfficientNet-B0. This highlights that training methodology can be as impactful as model scale.
Generalization to ImageNet-v2
To ensure gains weren't merely the result of overfitting the standard validation set, the authors evaluated performance on ImageNet-v2.
Figure 2 demonstrates that FixEfficientNet maintains its advantage on this more robust test set. While some state-of-the-art methods like Noisy Student show signs of overfitting, FixRes-enhanced models exhibit superior generalization across different data distributions.
State-of-the-Art Benchmarks
- ImageNet-2012: FixEfficientNet-L2 achieved 88.5% top-1 accuracy (single-crop), setting a new record.
- ImageNet Real Labels: On this "cleaned" benchmark, FixEfficientNet-L2 reached 90.9% top-1, demonstrating exceptional object identification capabilities.
- No Extra Data: Without massive external datasets, FixEfficientNet-B7 reached 85.7% top-1, a significant jump over the baseline EfficientNet.
Conclusion
FixEfficientNet demonstrates that addressing the fundamental train-test resolution discrepancy provides a substantial accuracy boost without increasing model complexity. By ensuring that the resolution scaling inherent in EfficientNet is handled correctly at both ends of the pipeline, the authors have established new benchmarks for parameter-efficient and high-performance image classification. The results on ImageNet-v2 and Real Labels further confirm the robustness of this approach.
Summary updated