No article text was recorded for this version.
multi-stage distillation overhead
Version 1 of 1164 papers164 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Mentor-KD: Making Small Language Models Better Multi-step Reasoners
- Blog: Parallel WaveNet: Fast High-Fidelity Speech Synthesis
- Blog: Distilling Object Detectors with Task Adaptive Regularization
- Blog: LightBSR: Towards Lightweight Blind Super-Resolution via Discriminative Implicit Degradation Representation Learning
- Blog: ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
- Blog: Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help?
- Blog: BERT-ERC: Fine-Tuning BERT Is Enough for Emotion Recognition in Conversation
- Blog: Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment
- Blog: Layer Compression of Deep Networks with Straight Flows
- Blog: Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
- Blog: DREAM: Efficient Dataset Distillation by Representative Matching
- Blog: SE-Bridge: Speech Enhancement with Consistent Brownian Bridge
- Blog: 3DG-STFM: 3D Geometric Guided Student-Teacher Feature Matching
- Blog: TSCAN: Context-Aware Uplift Modeling via Two-Stage Training for Online Merchant Business Diagnosis
- Blog: MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation
- Blog: Boosting Graph Neural Networks via Adaptive Knowledge Distillation
- Blog: In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation
- Blog: TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
- Blog: Differentiable JPEG-based Input Perturbation for Knowledge Distillation Amplification via Conditional Mutual Information Maximization
- Blog: Improved Knowledge Distillation via Teacher Assistant: Bridging the Gap Between Student and Teacher
- Blog: Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model
- Blog: Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization
- Blog: Exploring Target Representations for Masked Autoencoders
- Blog: Prototype-guided Cross-task Knowledge Distillation for Large-scale Models
- Blog: Improving Sequential Recommendation Consistency with Self-Supervised Imitation
- Blog: Visual Generation Without Guidance
- Blog: BPL: Bias-adaptive Preference Distillation Learning for Recommender System
- Blog: Revealing the Reciprocal Relations between Self-Supervised Stereo and Monocular Depth Estimation
- Blog: An improved YOLOv5s model for assessing apple graspability in automated harvesting scene
- Blog: Distilling Multiple Domains for Neural Machine Translation
- Blog: Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-Tailed Classification
- Blog: Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation
- Blog: Fast and Robust Visuomotor Riemannian Flow Matching Policy
- Blog: Rethinking the Stability-Plasticity Trade-off in Continual Learning from an Architectural Perspective
- Blog: R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis
- Blog: Fose: Fusion of One-Step Diffusion and End-to-End Network for Pansharpening
- Blog: Sparse2Dense: Learning to Densify 3D Features for 3D Object Detection
- Blog: One-Step Diffusion Distillation via Deep Equilibrium Models
- Blog: Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representations
- Blog: Memory-Distilled Selection for Noise-Robust Anomaly Detection
- Blog: Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
- Blog: Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation
- Blog: Style-Based Global Appearance Flow for Virtual Try-On
- Blog: One-Step Flow Policy Mirror Descent
- Blog: Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality
- Blog: OSMamba: Omnidirectional Spectral Mamba with Dual-Domain Prior Generator for Exposure Correction
- Blog: Importance Sampling Cams For Weakly-Supervised Segmentation
- Blog: Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
- Blog: Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation
- Blog: PromptMM: Multi-Modal Knowledge Distillation for Recommendation with Prompt-Tuning
- Blog: Regression Prior Networks
- Blog: ProReflow: Progressive Reflow with Decomposed Velocity
- Blog: Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition
- Blog: Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram
- Blog: CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
- Blog: Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation
- Blog: CAST: Contrastive Adaptation and Distillation for Semi-Supervised Instance Segmentation
- Blog: UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors
- Blog: Tackling Combinatorial Distribution Shift: A Matrix Completion Perspective
- Blog: Training Constrained Deconvolutional Networks for Road Scene Semantic Segmentation
Showing 60 of 164 papers.
History & citation
Current · Permanent link
- Multi-Stage Pipelines for Bridging Knowledge Distillation Gaps merged into this direction (69% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_multi_stage_distillation_overhead_e38935a2,
title = {multi-stage distillation overhead},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/multi-stage-distillation-overhead/rdr_e38935a259564a84a449366727246fe0},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_multi_stage_distillation_overhead_e38935a2_v1,
title = {multi-stage distillation overhead},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/multi-stage-distillation-overhead/rdr_e38935a259564a84a449366727246fe0/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}