No article text was recorded for this version.
multi-stage knowledge distillation
Version 2 of 2267 papers384 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better
- Blog: Densely Guided Knowledge Distillation using Multiple Teacher Assistants
- Blog: Improved Knowledge Distillation via Teacher Assistant: Bridging the Gap Between Student and Teacher
- Blog: Pacemaker: Intermediate Teacher Knowledge Distillation For On-The-Fly Convolutional Neural Network
- Blog: Densely Guided Knowledge Distillation using Multiple Teacher Assistants
- Blog: Improved Knowledge Distillation via Teacher Assistant
- Blog: Learning Lightweight Object Detectors via Progressive Knowledge Distillation
- Blog: Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
- Blog: PROD: Progressive Distillation for Dense Retrieval
- Blog: InDistill: Information flow-preserving knowledge distillation for model compression
- Blog: Towards Optimization-Friendly Binary Neural Network
- Blog: Neural TSP Solver with Progressive Distillation
- Blog: AMD: Automatic Multi-step Distillation of Large-scale Vision Models
- Blog: Croppable Knowledge Graph Embedding
- Blog: Knowledge Distillation via Route Constrained Optimization
- Blog: ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
- Blog: Building Accurate Simple Models with Multihop
- Blog: Capacity-Aware Federated Offline Reinforcement Learning for Resource-Heterogeneous Edge Clients
- Blog: QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation
- Blog: Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition
- Blog: LRH-Net: A Multi-Level Knowledge Distillation Approach for Low-Resource Heart Network
- Blog: LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation
- Blog: Towards Efficient Task-Driven Model Reprogramming with Foundation Models
- Blog: MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants
- Blog: HKD4VLM: A Progressive Hybrid Knowledge Distillation Framework for Robust Multimodal Hallucination and Factuality Detection in VLMs
- Blog: Boosting Graph Neural Networks via Adaptive Knowledge Distillation
- Blog: SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval
- Blog: TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
- Blog: LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters
- Blog: A Progressive Transformer for Unifying Binary Code Embedding and Knowledge Transfer
- Blog: SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought
- Blog: Adaptive Hierarchy-Branch Fusion for Online Knowledge Distillation
- Blog: Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning
- Blog: Distilling Spikes: Knowledge Distillation in Spiking Neural Networks
- Blog: Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
- Blog: Pacemaker: Intermediate Teacher Knowledge Distillation For On-The-Fly Convolutional Neural Network
- Blog: LANTERN: Scalable Distillation of Large Language Models for Job-Person Fit and Explanation
- Blog: Multi-stage Progressive Compression of Conformer Transducer for On-device Speech Recognition
- Blog: ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval
- Blog: TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
- Blog: Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
- Blog: ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques
- Blog: Boosting Residual Networks with Group Knowledge
- Blog: NutePrune: Efficient Progressive Pruning with Numerous Teachers for Large Language Models
- Blog: Task-agnostic Distillation of Encoder-Decoder Language Models
- Blog: MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
- Blog: Domain-invariant Progressive Knowledge Distillation for UAV-based Object Detection
- Blog: MiniRBT: A Two-stage Distilled Small Chinese Pre-trained Model
- Blog: MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval
- Blog: Progressive distillation induces an implicit curriculum
- Blog: Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation
- Blog: Boosting Large Language Models for Mental Manipulation Detection via Data Augmentation and Distillation
- Blog: LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation
- Blog: A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
- Blog: A Self-Distillation Recipe for Neural Machine Translation
- Blog: Boosting Graph Neural Networks via Adaptive Knowledge Distillation
- Blog: From Large to Tiny: Distilling and Refining Mathematical Expertise for Math Word Problems with Weakly Supervision
- Blog: Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation
- Blog: BridgeTA: Bridging the Representation Gap in Knowledge Distillation via Teacher Assistant for Bird's Eye View Map Segmentation
- Blog: SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval
Showing 60 of 267 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_multi_stage_knowledge_distillation_d66cf3ae,
title = {multi-stage knowledge distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/multi-stage-knowledge-distillation/rdr_d66cf3aea88e4f1a83609c695c3d9868},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 2, frozen text):
@misc{lacuna_multi_stage_knowledge_distillation_d66cf3ae_v2,
title = {multi-stage knowledge distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/multi-stage-knowledge-distillation/rdr_d66cf3aea88e4f1a83609c695c3d9868/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}