No article text was recorded for this version.
teacher-student capacity gap
Version 1 of 166 papers74 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Conditional generative data-free knowledge distillation
- Blog: Learn From the Past: Experience Ensemble Knowledge Distillation
- Blog: Self-Evolution Knowledge Distillation for LLM-based Machine Translation
- Blog: Knowledge Distillation from Multi-Modality to Single-Modality for Person Verification
- Blog: CasPoinTr: Point Cloud Completion with Cascaded Networks and Knowledge Distillation
- Blog: Prime-Aware Adaptive Distillation
- Blog: LLaVA-KD: A Framework of Distilling Multimodal Large Language Models
- Blog: Knowledge Distillation Based Recommendation Systems: A Comprehensive Survey
- Blog: Towards Optimization-Friendly Binary Neural Network
- Blog: Models Teaching Models: Improving Model Accuracy with Slingshot Learning
- Blog: Sample-level Adaptive Knowledge Distillation for Action Recognition
- Blog: Revisiting Knowledge Distillation for Autoregressive Language Models
- Blog: Unlocking SLM Potential for Data Analysis Code Generation via Non-Parametric Knowledge Distillation
- Blog: Self-Distillation: Towards Efficient and Compact Neural Networks
- Blog: AMFD: Distillation via Adaptive Multimodal Fusion for Multispectral Pedestrian Detection
- Blog: Distilling efficient Vision Transformers from CNNs for semantic segmentation
- Blog: TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation
- Blog: Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
- Blog: Knowledge Distillation Beyond Model Compression
- Blog: ResKD: Residual-Guided Knowledge Distillation
- Blog: CLIP-KD: An Empirical Study of CLIP Model Distillation
- Blog: Single Teacher, Multiple Perspectives: Teacher Knowledge Augmentation for Enhanced Knowledge Distillation
- Blog: Improving Consistency Models with Generator-Augmented Flows
- Blog: Distillation Scaling Laws
- Blog: Does Knowledge Distillation Really Work?
- Blog: Preserving Angles Improves Feature Distillation
- Blog: All You Need in Knowledge Distillation Is a Tailored Coordinate System
- Blog: Mind the Gap: Diverse NMT Models for Resource-Constrained Environments
- Blog: Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
- Blog: Self-supervised Models are Good Teaching Assistants for Vision Transformers
- Blog: Pretrained Speech Encoders and Efficient Fine-tuning Methods for Speech Translation: UPC at IWSLT 2022
- Blog: Small But Funny: A Feedback-Driven Approach to Humor Distillation
- Blog: Beyond Student: An Asymmetric Network for Neural Network Inheritance
- Blog: Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems
- Blog: Learning Lightweight Object Detectors via Progressive Knowledge Distillation
- Blog: Scaling Sparse and Dense Retrieval in Decoder-Only LLMs
- Blog: Tensor decomposition for minimization of E2E SLU model toward on-device processing
- Blog: PROD: Progressive Distillation for Dense Retrieval
- Blog: Teach Less, Learn More: On the Undistillable Classes in Knowledge Distillation
- Blog: Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better
- Blog: Knowledge Distillation Meets Few-Shot Learning: An Approach for Few-Shot Intent Classification Within and Across Domains
- Blog: Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup
- Blog: On Representation Knowledge Distillation for Graph Neural Networks
- Blog: SPENCER: Self-Adaptive Model Distillation for Efficient Code Retrieval
- Blog: Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression
- Blog: PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors
- Blog: Student-Teacher Learning From Clean Inputs to Noisy Inputs
- Blog: Aligning Sizes of Intermediate Layers by LoRA Adapter for Knowledge Distillation
- Blog: POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization
- Blog: Compact Neural Architecture Designs by Tensor Representations
- Blog: KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillation
- Blog: Tutoring Helps Students Learn Better: Improving Knowledge Distillation for BERT with Tutor Network
- Blog: Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
- Blog: Scaling Motion Forecasting Models with Ensemble Distillation
- Blog: QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillation
- Blog: T3DNet: Compressing Point Cloud Models for Lightweight 3-D Recognition
- Blog: Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Blog: Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation
- Blog: A Good Teacher Adapts Their Knowledge for Distillation
- Blog: Align Your Flow: Scaling Continuous-Time Flow Map Distillation
Showing 60 of 66 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_teacher_student_capacity_gap_0c19793e,
title = {teacher-student capacity gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/teacher-student-capacity-gap/rdr_0c19793e558043069cfc275b623f2acc},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 1, frozen text):
@misc{lacuna_teacher_student_capacity_gap_0c19793e_v1,
title = {teacher-student capacity gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/teacher-student-capacity-gap/rdr_0c19793e558043069cfc275b623f2acc/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}