Dual Information Purification for Lightweight SAR Object Detection

Xi Yang, Jiachen Sun, Songsong Duan, De ChengAAAI 2025 · DOI

TL;DR

This paper introduces DIPKD, a novel knowledge distillation framework designed to improve lightweight SAR object detection. By purifying the teacher model's knowledge through selective noise suppression and feature decoupling, DIPKD significantly boosts student model performance, even surpassing the teacher in some cases.

What Problem Does This Paper Address?

Synthetic Aperture Radar (SAR) imagery is crucial for all-weather earth observation, enabling object detection for applications like maritime and air domain monitoring. While deep learning models excel at SAR object detection, their computational demands hinder real-time applications. Knowledge Distillation (KD) offers a solution by training smaller, faster "student" models to mimic larger, more accurate "teacher" models. However, existing KD methods struggle with SAR imagery due to its inherent speckle noise and high background-to-target ratio. This noise can obscure key features and mislead the distillation process, leading to suboptimal student models that fail to capture essential information.

Methods

The proposed Dual Information Purification Knowledge Distillation (DIPKD) method tackles these challenges with three core components:

  • Selective Noise Suppression (SNS): This module employs spatial and channel attention-guided wavelet denoising to reduce speckle noise in global features. By focusing on critical feature areas, it preserves essential information while filtering out misleading noise from the teacher model.
  • Knowledge Level Decoupling (KLD): This module separates features into "target" and "non-target" knowledge. This decoupling helps balance feature mapping, reduces background noise interference, and allows the student model to better learn and retain critical information, especially in scenarios with a high background-to-target ratio.
  • Reverse Information Transfer (RIT): This branch uses the student model's predicted output to refine intermediate features. It subtly reduces the teacher model's dominance, encouraging the student to explore its potential more fully and compensating for any loss of detailed local information.

These modules work in concert to enhance the purity and efficacy of the transferred knowledge, leading to more refined and accurate object detection.

Datasets

The authors evaluated their method on two standard SAR object detection datasets:

  • SSDD (Ship Detection Dataset): A dataset specifically for ship detection in SAR images.
  • HRSID (High-Resolution SAR Images Dataset): Another dataset focused on ship detection in high-resolution SAR imagery.

Key Insights and Ideas

  • Speckle Noise is a Major Hurdle for KD in SAR: The paper strongly emphasizes that the unique speckle noise in SAR images significantly degrades the purity of knowledge transferred during distillation, a problem often overlooked by methods designed for natural images.
  • Decoupling Target and Non-Target Knowledge is Crucial: The high background-to-target ratio in SAR images means that simply distilling all features can overwhelm the student with irrelevant information. Separating target and non-target knowledge allows for more focused learning.
  • Reverse Information Transfer Empowers the Student: Instead of solely relying on the teacher, the RIT module allows the student to leverage its own predictions to guide its learning, leading to more independent and potentially superior feature extraction.
  • Student Models Can Surpass Teachers: The results demonstrate that a well-guided student model, through effective knowledge purification, can not only match but even exceed the performance of its teacher model on certain metrics.

Key Results

DIPKD significantly outperforms existing distillation techniques on both SSDD and HRSID datasets. Specifically:

  • The student model achieved mAP scores of 60.2% on SSDD and 51.4% on HRSID.
  • Compared to the teacher model (ResNet50 backbone), the student model (ResNet18 backbone) showed performance improvements of 1.3% and 2.9% on SSDD and HRSID respectively, highlighting the effectiveness of the purification approach.
  • Ablation studies showed that each component (SNS, KLD, RIT) individually improved performance, with RIT providing the largest boost. Combined, they achieved the highest overall improvement.

Limitations and Future Work

  • Hyperparameter Sensitivity: While the paper notes robustness to the temperature hyperparameter ($\tau$), the optimal settings for $\alpha$, $\beta$, and $\gamma$ (balancing the different loss components) are determined empirically. Further research could explore more adaptive or automated methods for setting these parameters.
  • Future Focus on Knowledge Acquisition: The authors mention that optimizing knowledge acquisition remains a future focus, suggesting potential avenues for further research into more efficient and effective ways for students to learn from teachers.

Summary updated

BibTeX
@inproceedings{yang2025dual,
  author = {Xi Yang and Jiachen Sun and Songsong Duan and De Cheng},
  title = {{Dual Information Purification for Lightweight SAR Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.33004},
}
Figure