Abstract
Deep neural networks often fail at Open-Set Recognition (OSR) because they overfit to high-frequency textures, leading to overconfident predictions on unknown, out-of-distribution classes. This paper introduces FreSH, a framework that uses wavelet transforms to separate image frequencies, shuffling high-frequency details while enhancing low-frequency global structures. By correcting this inductive bias, FreSH achieves state-of-the-art OSR performance without requiring extra parameters or complex auxiliary networks.
Introduction: The Challenge of Open-Set Recognition
In real-world deployments, computer vision models inevitably encounter objects they were never trained on. Open-Set Recognition (OSR) is the task of accurately classifying known classes while reliably rejecting these unknown classes.
Standard Deep Neural Networks (DNNs) struggle with OSR because they tend to be catastrophically overconfident when predicting unknown classes. The authors trace this flaw to an inductive bias in spatial-domain training: DNNs tend to over-perceive subtle, high-frequency (HF) local textures rather than focusing on low-frequency (LF) global structures.
Because textures and structures are deeply coupled in standard spatial images, a model might confidently misclassify an unknown object simply because it shares a similar texture with a known class. As illustrated in the figure above, even minor damage to high-frequency bands can cause a model to flip its prediction or fail to recognize a known class, while humans remain unaffected. This proves that CNNs are dangerously over-reliant on high-frequency textures.
Methodology: Frequency Shuffling and Enhancement (FreSH)
To break the coupling between texture and structure, the authors propose FreSH (Frequency Shuffling and Enhancement), which operates in the frequency domain using the Discrete Wavelet Transform (DWT) with Haar filters. FreSH consists of two main components:
- High-Frequency Shuffling (HFS): Used exclusively during training, HFS acts as a novel data augmentation strategy. It takes an image, applies a random transformation, and extracts the HF subbands. It then swaps these new HF subbands with the original image's HF subbands and reconstructs the image via Inverse DWT. By constantly perturbing the texture details, HFS forces the network to stop overfitting to high-frequency features and learn robust, low-frequency invariance.
- Low-Frequency Residual (LFR) Learning: To explicitly force the network to prioritize global structure, the authors introduce a skip-connection protocol. They extract the LF subbands from shallow convolutional layers and feed them directly into deeper layers. This acts as a residual connection that emphasizes global shape and structure throughout the network's feature extraction process.
Experimental Results
FreSH was evaluated on standard OSR benchmarks of varying complexity, including MNIST, SVHN, CIFAR, and Tiny-ImageNet. The results demonstrate that managing frequency processing is a highly effective way to calibrate model confidence without the need for complex generative models (like GANs or Autoencoders) to synthesize "fake" unknown samples.
Key Performance Insights
- Unknown Detection: FreSH achieves state-of-the-art AUROC across almost all benchmarks. Most notably, on the highly challenging Tiny-ImageNet dataset, it reaches 83.8% AUROC, significantly outperforming previous methods.
- Open-Set Classification Rate (OSCR): OSCR measures the trade-off between detecting unknowns and correctly classifying knowns. FreSH pushes the SOTA forward by a massive +10.1% margin on Tiny-ImageNet (reaching 77.9%).
- Closed-Set Accuracy: Rejecting unknowns usually comes at the cost of closed-set accuracy. FreSH actually improves closed-set accuracy (e.g., a +3.1% gain on Tiny-ImageNet), proving that focusing on global structure makes representations more discriminative overall.
- Efficiency: FreSH achieves these results using a standard VGG32 backbone, requiring a fraction of the parameters used by competing generative-based OSR methods.
Discussion and Conclusion
FreSH demonstrates that the "Aha!" moment—recognizing that CNNs are over-reliant on high-frequency textures—can be translated into a practical, parameter-free framework. While previous state-of-the-art methods relied on complex generative models to synthesize unknown samples, FreSH proves that frequency decoupling is a more efficient and robust alternative.
Future work will explore generalized frequency representations to tackle even more challenging visual tasks beyond standard image classification. While the current use of Haar wavelets is effective, exploring more flexible frequency decomposition rules could further align the method with complex semantic boundaries in diverse datasets.
Summary updated