No article text was recorded for this version.
distillation-driven regularization
Version 2 of 2209 papers243 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
- Blog: Reverse Knowledge Distillation: Training a Large Model Using a Small One for Retinal Image Matching on Limited Data
- Blog: On the benefits of knowledge distillation for adversarial robustness
- Blog: Do Deep Nets Really Need to be Deep?
- Blog: SUOD: Toward Scalable Unsupervised Outlier Detection
- Blog: On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective
- Blog: Knowledge Distillation: The Functional Perspective
- Blog: Does Knowledge Distillation Really Work?
- Blog: Knowledge Distillation For Recurrent Neural Network Language Modeling With Trust Regularization
- Blog: Analysis of Knowledge Transfer in Kernel Regime
- Blog: Solvable Model for Inheriting the Regularization through Knowledge Distillation
- Blog: Teacher-student training improves the accuracy and efficiency of machine learning interatomic potentials
- Blog: Understanding Self-Distillation in the Presence of Label Noise
- Blog: Toward Better Target Representation for Source-Free and Black-Box Domain Adaptation
- Blog: Learning to Project for Cross-Task Knowledge Distillation
- Blog: Distilling Textual Priors from LLM to Efficient Image Fusion
- Blog: Distilling Object Detectors with Global Knowledge
- Blog: Transformer to CNN: Label-scarce distillation for efficient text classification
- Blog: Distilling Knowledge From a Deep Pose Regressor Network
- Blog: Distilling foundation models for robust and efficient models in digital pathology
- Blog: Revisiting Intermediate Layer Distillation for Compressing Language Models: An Overfitting Perspective
- Blog: Knowledge Distillation for Oriented Object Detection on Aerial Images
- Blog: KD-DLGAN: Data Limited Image Generation via Knowledge Distillation
- Blog: Solvable Model for Inheriting the Regularization through Knowledge Distillation
- Blog: P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection
- Blog: Explaining Sequence-Level Knowledge Distillation as Data-Augmentation for Neural Machine Translation
- Blog: Efficient Vision Transformers via Fine-Grained Manifold Distillation
- Blog: Revisiting Knowledge Distillation via Label Smoothing Regularization
- Blog: In defense of dual-encoders for neural ranking
- Blog: DSD²: Can We Dodge Sparse Double Descent and Compress the Neural Network Worry-Free?
- Blog: DE-RRD: A Knowledge Distillation Framework for Recommender System
- Blog: Adversarial local distribution regularization for knowledge distillation
- Blog: A Comprehensive Overhaul of Feature Distillation
- Blog: Distilling the Knowledge of Romanian BERTs Using Multiple Teachers
- Blog: iDAT: inverse Distillation Adapter-Tuning
- Blog: The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
- Blog: Perspective-Aware Teaching: Adapting Knowledge for Heterogeneous Distillation
- Blog: Distilling Object Detectors with Task Adaptive Regularization
- Blog: Transferring knowledge from a RNN to a DNN
- Blog: Understanding Self-Distillation in the Presence of Label Noise
- Blog: Dodging the Sparse Double Descent
- Blog: On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving
- Blog: Improving Fast Segmentation With Teacher-Student Learning
- Blog: Co-training and Co-distillation for Quality Improvement and Compression of Language Models
- Blog: daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
- Blog: Attention-Guided Answer Distillation for Machine Reading Comprehension
- Blog: RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation
- Blog: On student-teacher deviations in distillation: does it pay to disobey?
- Blog: Towards Understanding Ensemble Distillation in Federated Learning
- Blog: Why Are Distribution-Matching Distilled Students Lazy? Understanding the Copying Behavior in Few-Step Distillation
- Blog: JEDI: Joint Expert Distillation in a Semi-Supervised Multi-Dataset Student-Teacher Scenario for Video Action Recognition
- Blog: Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification
- Blog: Cross-Layer Distillation with Semantic Calibration
- Blog: RAIN: RegulArization on Input and Network for Black-Box Domain Adaptation
- Blog: Regularizing Dialogue Generation by Imitating Implicit Scenarios
- Blog: Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers
- Blog: Self-Guidance: Improve Deep Neural Network Generalization via Knowledge Distillation
- Blog: TTT-KD: Test-Time Training for 3D Semantic Segmentation through Knowledge Distillation from Foundation Models
- Blog: Diffusion Domain Teacher: Diffusion Guided Domain Adaptive Object Detector
- Blog: Unsupervised Explanation Generation for Machine Reading Comprehension
Showing 60 of 209 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_distillation_driven_regularization_c7a1d85a,
title = {distillation-driven regularization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/distillation-driven-regularization/rdr_c7a1d85a35424b328bbb96792a41442f},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 2, frozen text):
@misc{lacuna_distillation_driven_regularization_c7a1d85a_v2,
title = {distillation-driven regularization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/distillation-driven-regularization/rdr_c7a1d85a35424b328bbb96792a41442f/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}