No article text was recorded for this version.
attention map distillation
Version 1 of 1295 papers407 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Class Attention Transfer Based Knowledge Distillation
- Blog: Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
- Blog: Focal and Global Knowledge Distillation for Detectors
- Blog: Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
- Blog: Class Attention Transfer Based Knowledge Distillation
- Blog: Attention Distillation for Learning Video Representations
- Blog: CompoDistill: Attention Distillation for Compositional Reasoning in Multimodal LLMs
- Blog: Bridging the Gap in Missing Modalities: Leveraging Knowledge Distillation and Style Matching for Brain Tumor Segmentation
- Blog: Paying More Attention to Motion: Attention Distillation for Learning Video Representations
- Blog: Cosine similarity-guided knowledge distillation for robust object detectors
- Blog: Attention Distillation: self-supervised vision transformer students need more guidance
- Blog: Label-guided Attention Distillation for lane segmentation
- Blog: Multimodal Transformer Distillation for Audio-Visual Synchronization
- Blog: On the Surprising Effectiveness of Attention Transfer for Vision Transformers
- Blog: Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching
- Blog: Label-guided Attention Distillation for Lane Segmentation
- Blog: No Head Left Behind - Multi-Head Alignment Distillation for Transformers
- Blog: LoLCATs: On Low-Rank Linearizing of Large Language Models
- Blog: Infrared UAV Target Tracking with Dynamic Feature Refinement and Global Contextual Attention Knowledge Distillation
- Blog: An Attention-based Representation Distillation Baseline for Multi-Label Continual Learning
- Blog: Student-Oriented Teacher Knowledge Refinement for Knowledge Distillation
- Blog: LoReTrack: Efficient and Accurate Low-Resolution Transformer Tracking
- Blog: Enhance Long Text Understanding via Distilled Gist Detector from Abstractive Summarization
- Blog: Conditional Generative Data-Free Knowledge Distillation based on Attention Transfer
- Blog: GCISG: Guided Causal Invariant Learning for Improved Syn-to-Real Generalization
- Blog: Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation
- Blog: Attention-Guided Answer Distillation for Machine Reading Comprehension
- Blog: Moonshine: Distilling with Cheap Convolutions
- Blog: Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation
- Blog: Attention Distillation for Detection Transformers: Application to Real-Time Video Object Detection in Ultrasound
- Blog: How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
- Blog: Towards exploring continual learning for toxicologic pathology in pharmaceutical drug discovery
- Blog: Source-Free Domain Adaptation for Semantic Segmentation
- Blog: Distilling with Performance Enhanced Students
- Blog: Self-supervised Video Object Segmentation with Distillation Learning of Deformable Attention
- Blog: Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features
- Blog: An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-training
- Blog: Transferring Knowledge with Attention Distillation for Multi-Domain Image-to-Image Translation
- Blog: Creating Lightweight Object Detectors with Model Compression for Deployment on Edge Devices
- Blog: How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
- Blog: Distilled Siamese Networks for Visual Tracking
- Blog: DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection
- Blog: Focal and Global Knowledge Distillation for Detectors
- Blog: Local-Selective Feature Distillation for Single Image Super-Resolution
- Blog: Observation, Analysis, and Solution: Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training
- Blog: VIRT: Improving Representation-based Models for Text Matching through Virtual Interaction
- Blog: Explainable Knowledge Distillation for On-device Chest X-Ray Classification
- Blog: Pre-training Attention Mechanisms
- Blog: How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
- Blog: Attention Distillation: A Unified Approach to Visual Characteristics Transfer
- Blog: CanKD: Cross-Attention-based Non-local Operation for Feature-based Knowledge Distillation
- Blog: Federated Learning With Privacy-Preserving Ensemble Attention Distillation
- Blog: Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors
- Blog: Region-aware Knowledge Distillation for Efficient Image-to-Image Translation
- Blog: SAM-COD: SAM-guided Unified Framework for Weakly-Supervised Camouflaged Object Detection
- Blog: Low-Resolution Chest X-ray Classification via Knowledge Distillation and Multi-task Learning
- Blog: Multi-Type Self-Attention Guided Degraded Saliency Detection
- Blog: AttTrack: Online Deep Attention Transfer for Multi-Object Tracking
- Blog: A Closer Look at Self-supervised Lightweight Vision Transformers
- Blog: Do not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting
Showing 60 of 295 papers.
History & citation
Current · Permanent link
- Knowledge Distillation via Attention Map Alignment merged into this direction (69% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_attention_map_distillation_df0719bb,
title = {attention map distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-map-distillation/rdr_df0719bbbd2e4cbdbc16115b0bcb2a12},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_attention_map_distillation_df0719bb_v1,
title = {attention map distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-map-distillation/rdr_df0719bbbd2e4cbdbc16115b0bcb2a12/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}