Abstract
This work introduces a novel biorthogonal tunable wavelet unit constructed using a lifting scheme that relaxes both the orthogonality and equal filter length constraints, providing greater flexibility in filter design. The proposed unit enhances convolution, pooling, and downsampling operations, leading to improved image classification and anomaly detection in convolutional neural networks (CNN). When integrated into an 18-layer residual neural network (ResNet-18), the approach improved classification accuracy on CIFAR-10 by 2.12% and on the Describable Textures Dataset (DTD) by 9.73%, demonstrating its effectiveness in capturing fine-grained details. Similar improvements were observed in ResNet-34. For anomaly detection in the hazelnut category of the MVTec Anomaly Detection dataset, the proposed method achieved competitive and wellbalanced performance in both segmentation and detection tasks, outperforming existing approaches in terms of accuracy and robustness.
Introduction
Standard Convolutional Neural Networks (CNNs) typically rely on operations like max pooling or strided convolutions to reduce spatial dimensions. While computationally efficient, these methods are inherently "lossy," often discarding fine-grained, high-frequency details in favor of dominant features. This loss of information, known as aliasing, can be detrimental in tasks requiring high precision, such as texture classification or anomaly detection.
While previous research has explored Wavelet Transforms to mitigate this by decomposing images into frequency components, traditional wavelet-based CNNs are often constrained by rigid mathematical requirements like orthogonality and equal filter lengths. These constraints limit the network's ability to adapt filters to specific dataset characteristics.
The LS-BiorUwU Framework
To address these limitations, we propose the LS-BiorUwU (Lifting Scheme Biorthogonal Tunable Wavelet Unit). This unit utilizes a Lifting Scheme to construct biorthogonal wavelets, offering several advantages:
- Relaxed Constraints: Filters are no longer required to be orthogonal or of equal length, allowing for significantly more flexible filter design.
- Tunability: The parameters of the lifting steps are learnable, enabling the network to optimize wavelet filters during training for the specific data at hand.
- Information Preservation: Unlike traditional pooling, LS-BiorUwU preserves all frequency components—low-low (LL), low-high (LH), high-low (HL), and high-high (HH). These components are processed by a single-layer Fully Connected Network (FCN) to determine the optimal feature map.
As shown in the figure above, the LS-BiorUwU unit is designed to seamlessly replace standard downsampling operations, including max pooling and strided convolutions, within existing architectures like ResNet.
Experimental Results
The effectiveness of the LS-BiorUwU unit was evaluated across three diverse datasets: CIFAR-10 (low-resolution objects), the Describable Textures Dataset (DTD, high-resolution textures), and MVTec AD (anomaly detection in hazelnuts).
Performance Gains
The integration of LS-BiorUwU into ResNet-18 yielded significant performance improvements:
- DTD Accuracy: Increased from 33.99% to 43.72% (a 9.73% absolute gain).
- CIFAR-10 Accuracy: Improved from 92.44% to 94.56% (a 2.12% gain).
- Anomaly Detection: On the MVTec hazelnut dataset, the model achieved a 97.21% SegAUROC, outperforming both the baseline and previous orthogonal wavelet models.
The visual comparison above highlights why the wavelet approach is so effective. In texture-heavy datasets like DTD, the high-frequency components (LH, HL, HH) contain critical information that is often lost in standard downsampling but preserved by the LS-BiorUwU unit.
Conclusion
By relaxing the mathematical constraints of traditional wavelets and introducing learnable lifting steps, the LS-BiorUwU unit provides a powerful alternative to standard pooling in CNNs. The substantial gains in texture classification and anomaly detection demonstrate the importance of preserving high-frequency details. Future work will explore dual lifting schemes and stopband-attenuation constraints to further refine filter selectivity and performance.
Summary updated