No article text was recorded for this version.
data-efficient knowledge distillation
Version 2 of 2171 papers207 conceptsfrozen 2026-10-04
Papers in this version
- Blog: In Teacher We Trust: Learning Compressed Models for Pedestrian Detection
- Blog: Subclass Distillation
- Blog: Scaling Laws for Data-Efficient Visual Transfer Learning
- Blog: Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget
- Blog: Locally Linear Region Knowledge Distillation
- Blog: Contact-Distil: Boosting Low Homologous Protein Contact Map Prediction by Self-Supervised Distillation
- Blog: Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
- Blog: Knowledge Distillation from Internal Representations
- Blog: Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget
- Blog: Knowledge Distillation for Throat Microphone Speech Recognition
- Blog: Data-Efficient Ranking Distillation for Image Retrieval
- Blog: Data-Efficient Low-Complexity Acoustic Scene Classification via Distilling and Progressive Pruning
- Blog: Learning Student Networks with Few Data
- Blog: Distilling Transformers into Simple Neural Networks with Unlabeled Transfer Data
- Blog: A Tale of LLMs and Induced Small Proxies: Scalable Agents for Knowledge Mining
- Blog: Knowledge Distillation Meets Few-Shot Learning: An Approach for Few-Shot Intent Classification Within and Across Domains
- Blog: DisCo: Effective Knowledge Distillation For Contrastive Learning of Sentence Embeddings
- Blog: Skill-Aware Data Selection and Fine-Tuning for Data-Efficient Reasoning Distillation
- Blog: Causal Distillation for Language Models
- Blog: TMLKD: Few-shot Trajectory Metric Learning via Knowledge Distillation
- Blog: Knowledge Transfer with Jacobian Matching
- Blog: Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
- Blog: Linear Projections of Teacher Embeddings for Few-Class Distillation
- Blog: Augmentation with Projection: Towards an Effective and Efficient Data Augmentation Paradigm for Distillation
- Blog: MKD: Mixup-Based Knowledge Distillation for Mandarin End-to-End Speech Recognition
- Blog: Knowledge-Distilled Graph Neural Networks for Personalized Epileptic Seizure Detection
- Blog: Subclass Knowledge Distillation with Known Subclass Labels
- Blog: Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
- Blog: Layerwise Bregman Representation Learning of Neural Networks with Applications to Knowledge Distillation
- Blog: Canonical Latent Representations in Conditional Diffusion Models
- Blog: View-aware Cross-modal Distillation for Multi-view Action Recognition
- Blog: Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
- Blog: Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
- Blog: DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detection
- Blog: Distilling EEG representations via capsules for affective computing
- Blog: Compressing Visual-linguistic Model via Knowledge Distillation
- Blog: Distilling a Pretrained Language Model to a Multilingual ASR Model
- Blog: Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation
- Blog: Rationale-Guided Distillation for E-Commerce Relevance Classification: Bridging Large Language Models and Lightweight Cross-Encoders
- Blog: Well-Read Students Learn Better: The Impact of Student Initialization on Knowledge Distillation
- Blog: Teacher Guided Training: An Efficient Framework for Knowledge Transfer
- Blog: Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model
- Blog: CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition
- Blog: Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers
- Blog: TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
- Blog: Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax
- Blog: Evaluating distillation methods for data-efficient syntax learning
- Blog: Unifying distillation and privileged information David Lopez-Paz
- Blog: Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation
- Blog: Data Distillation: A Survey
- Blog: Reinforced Multi-Teacher Selection for Knowledge Distillation
- Blog: Low-resource Low-footprint Wake-word Detection using Knowledge Distillation
- Blog: Towards Understanding Knowledge Distillation
- Blog: Where to Begin? From Random to Foundation Model Instructed Initialization in Federated Learning for Medical Image Segmentation
- Blog: Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation
- Blog: Variational Information Distillation for Knowledge Transfer
- Blog: Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanations
- Blog: Correction to: Knowledge distillation for efficient standard scanplane detection of fetal ultrasound
- Blog: Condensed Sample-Guided Model Inversion for Knowledge Distillation
- Blog: EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM
Showing 60 of 171 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_data_efficient_knowledge_distillation_2b15fdcd,
title = {data-efficient knowledge distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/data-efficient-knowledge-distillation/rdr_2b15fdcd1e28487a9a49910ce0138a4c},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 2, frozen text):
@misc{lacuna_data_efficient_knowledge_distillation_2b15fdcd_v2,
title = {data-efficient knowledge distillation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/data-efficient-knowledge-distillation/rdr_2b15fdcd1e28487a9a49910ce0138a4c/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}