No article text was recorded for this version.
self-distillation
Version 1 of 1149 papers188 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Self-Distilled Self-supervised Representation Learning
- Blog: Self-Distillation Based on High-level Information Supervision for Compressing End-to-End ASR Model
- Blog: D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement
- Blog: Self-Teaching Networks
- Blog: Self-Distillation: Towards Efficient and Compact Neural Networks
- Blog: Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
- Blog: Self-Guidance: Improve Deep Neural Network Generalization via Knowledge Distillation
- Blog: Graph-based Knowledge Distillation: A survey and experimental evaluation
- Blog: Universally Slimmable Networks and Improved Training Techniques
- Blog: Cross-modal Features Interaction-and-Aggregation Network with Self-consistency Training for Speech Emotion Recognition
- Blog: Domain-Agnostic Clustering with Self-Distillation
- Blog: Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition
- Blog: One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model
- Blog: SDNet: An Extremely Efficient Portrait Matting Model via Self-Distillation
- Blog: Slimmed Asymmetrical Contrastive Learning and Cross Distillation for Lightweight Model Training
- Blog: MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation
- Blog: Feature Interaction Fusion Self-Distillation Network For CTR Prediction
- Blog: Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
- Blog: Golden Cudgel Network for Real-Time Semantic Segmentation
- Blog: Self-Supervised Speech Representation Learning: A Review
- Blog: D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement
- Blog: SalNAS: Efficient Saliency-prediction Neural Architecture Search with self-knowledge distillation
- Blog: Beyond External Guidance: Unleashing the Semantic Richness Inside Diffusion Transformers for Improved Training
- Blog: Image-Goal Navigation Using Refined Feature Guidance and Scene Graph Enhancement
- Blog: FastBERT: a Self-distilling BERT with Adaptive Inference Time
- Blog: Learning From Yourself: A Self-Distillation Method for Fake Speech Detection
- Blog: Self-Regulated Feature Learning via Teacher-free Feature Distillation
- Blog: Learning Lightweight Lane Detection CNNs by Self Attention Distillation
- Blog: Self-knowledge distillation for fine-grained image classification
- Blog: Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models
- Blog: Multi-target Knowledge Distillation via Student Self-reflection
- Blog: Improved Techniques for Training Adaptive Deep Networks
- Blog: Distillation-Based Training for Multi-Exit Architectures
- Blog: Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillation
- Blog: Self-Referenced Deep Learning
- Blog: Extracting knowledge from features with multilevel abstraction
- Blog: Data-Distortion Guided Self-Distillation for Deep Neural Networks
- Blog: Long Short-Term Sample Distillation
- Blog: Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement
- Blog: Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency
- Blog: Generalizing to Unseen Domains in Diabetic Retinopathy Classification
- Blog: FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
- Blog: Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
- Blog: Layer-to-Layer Knowledge Mixing in Graph Neural Network for Chemical Property Prediction
- Blog: Layer-wise Regularized Dropout for Neural Language Models
- Blog: Universally Slimmable Networks and Improved Training Techniques
- Blog: Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing
- Blog: EPSD: Early Pruning with Self-Distillation for Efficient Model Compression
- Blog: Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation
- Blog: Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification
- Blog: Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clients
- Blog: Self-Referenced Deep Learning
- Blog: Adaptive Regularization of Labels
- Blog: Self-Regulation for Semantic Segmentation
- Blog: Cyclic Differentiable Architecture Search
- Blog: Editorial: Efficient deep neural network for intelligent robot system: Focusing on visual signal processing
- Blog: On Self-Distilling Graph Neural Network
- Blog: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis
- Blog: Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
- Blog: EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
Showing 60 of 149 papers.
History & citation
Current · Permanent link
- Internal Self-Distillation for Efficient Neural Network Training merged into this direction (52% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_self_distillation_c4aee043,
title = {self-distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/self-distillation/rdr_c4aee0438cac435bbf4ec94bcdf1c95a},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_self_distillation_c4aee043_v1,
title = {self-distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/self-distillation/rdr_c4aee0438cac435bbf4ec94bcdf1c95a/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}