Abstract
Modern semantic segmentation models often prioritize high-level contextual information at the expense of fine-grained, pixel-level details. This trade-off is particularly evident in compact models designed for edge deployment, which frequently produce coarse boundaries and blurry masks. This paper presents a novel framework, Structural and Statistical Texture Knowledge Distillation (SSTKD), designed to extract and transfer dual-natured texture knowledge from large teacher models to efficient student models. By leveraging Contourlet Decomposition and Texture Intensity Equalization, the proposed method significantly enhances segmentation accuracy across various benchmarks, including ultra-high-resolution (UHR) tasks, without substantial computational overhead.
Introduction
Semantic segmentation requires a precise understanding of both global context and local detail. While deep convolutional neural networks (CNNs) have mastered the extraction of high-level semantic features, they often struggle with "fine print" details such as boundaries, smoothness, and local color contrast. This leads to two primary challenges: the inherent conflict between model efficiency and boundary accuracy, and the tendency of existing Knowledge Distillation (KD) techniques to overlook low-level texture features.
To bridge this gap, we introduce SSTKD, a framework that treats texture as a two-sided coin—comprising both structural (local geometric) and statistical (global distribution) components. By distilling these specific knowledge types, we enable compact models to achieve performance levels previously reserved for much larger architectures.
Methodology
Structural and Statistical Texture Knowledge Distillation (SSTKD)
The SSTKD framework forces a student model to mimic the texture-rich early layers of a teacher model through two specialized modules and a custom loss function.
- Contourlet Decomposition Module (CDM): This module targets structural texture, such as local lines and edges. Unlike standard spatial filters or wavelets—which are effective for point discontinuities but struggle with curves—the CDM utilizes a contourlet transform. By combining Laplacian pyramids with directional filter banks, the CDM captures smooth contours and geometric patterns with high efficiency, providing the student model with a superior representation of object boundaries.
- Texture Intensity Equalization Module (TIEM): This module addresses statistical texture, focusing on global properties like intensity distributions and contrast. Inspired by classic Histogram Equalization, TIEM quantizes feature maps and constructs a differentiable "histogram" of feature intensities. It then employs graph reasoning to denoise and equalize these distributions, allowing the model to better distinguish between subtle variations in image intensity.
- Quantization Congruence Loss (QCL): To ensure effective knowledge transfer, the QCL function is employed to align the statistical texture distributions between the teacher and student models, ensuring the student learns the underlying intensity patterns of the teacher's feature space.
Standalone Architectures
Beyond distillation, the authors integrated these modules into standalone architectures: STLNet++ for standard semantic segmentation and U-SSNet for Ultra-High-Resolution (UHR) images. These models demonstrate that explicit texture learning improves base performance even in the absence of a teacher model.
Experimental Setup
The effectiveness of SSTKD and its derivative architectures was evaluated across seven benchmarks covering three distinct tasks:
- Knowledge Distillation: Cityscapes, ADE20K, and Pascal VOC 2012.
- Generic Semantic Segmentation: Cityscapes, Pascal Context, and ADE20K.
- Ultra-High-Resolution (UHR) Segmentation: DeepGlobe, Inria Aerial, and URUR.
Results and Discussion
Distillation Performance
The impact of SSTKD on compact models is substantial. On the Cityscapes dataset, a baseline ResNet18 student model achieved an initial 69.10% mIoU. While standard distillation techniques improved this to 72.47%, the application of SSTKD (combining both structural and statistical distillation) pushed the performance to 76.65%. This result effectively closed the gap with the ResNet101 teacher model (78.56%) to within 1.91%, demonstrating that texture knowledge is a critical missing component in traditional KD.
UHR and Efficiency
In the realm of UHR segmentation, the U-SSNet variant achieved state-of-the-art (SOTA) results, reaching 76.9% mIoU on DeepGlobe and 77.5% on Inria Aerial. Notably, these gains were achieved with high efficiency; the CDM and TIEM modules add only 1.24M and 2.80M parameters, respectively, making them suitable for deployment on resource-constrained edge devices.
Synergistic Effects
A key insight from the study is the synergy between structural and statistical learning. Enhancing local edges (structural) assists the model in understanding global contrast (statistical), and vice versa. Furthermore, the use of contourlets proved superior to traditional wavelets or Fast Fourier Transforms (FFT) for representing the smooth, continuous curves typical of real-world object boundaries.
Conclusion and Future Work
SSTKD provides a robust framework for improving semantic segmentation by focusing on the often-neglected domain of low-level texture. While highly effective for CNNs, the authors note that the approach faces challenges when applied to Vision Transformers (ViTs). The patch-based nature of ViTs tends to fracture continuous textures, limiting the effectiveness of global intensity equalization.
Future research will focus on adapting these texture-learning modules for patch-based architectures and large vision-language models. Additionally, the authors plan to explore the application of texture distillation in other dense prediction tasks, such as depth estimation and surface normal prediction.
Summary updated