No article text was recorded for this version.
single-layer distillation bottleneck
Version 2 of 2164 papers167 conceptsfrozen 2026-10-04
Papers in this version
- Blog: GUIDE: Guided Initialization and Distillation of Embeddings
- Blog: MTA4DPR: Multi-Teaching-Assistants Based Iterative Knowledge Distillation for Dense Passage Retrieval
- Blog: Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
- Blog: DE-RRD: A Knowledge Distillation Framework for Recommender System
- Blog: Lightweight Modality Adaptation to Sequential Recommendation via Correlation Supervision
- Blog: A Comprehensive Overhaul of Feature Distillation
- Blog: VkD: Improving Knowledge Distillation using Orthogonal Projections
- Blog: Distilling EEG representations via capsules for affective computing
- Blog: Adversarial Network Compression
- Blog: ARDIR: Improving Robustness using Knowledge Distillation of Internal Representation
- Blog: TINED: GNNs-to-MLPs by Teacher Injection and Dirichlet Energy Distillation
- Blog: UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation
- Blog: Beyond Output Matching: Bidirectional Alignment for Enhanced In-Context Learning
- Blog: Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures
- Blog: Model Compression Using Optimal Transport
- Blog: Hierarchical Self-supervised Augmented Knowledge Distillation
- Blog: UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations
- Blog: Speech Emotion Recognition via Multi-Level Cross-Modal Distillation
- Blog: SKIntern: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models
- Blog: Masked Autoencoders Enable Efficient Knowledge Distillers
- Blog: Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment
- Blog: Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
- Blog: HAD-Net: A Hierarchical Adversarial Knowledge Distillation Network for Improved Enhanced Tumour Segmentation Without Post-Contrast Images
- Blog: Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
- Blog: Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
- Blog: Representational Distance Learning for Deep Neural Networks
- Blog: Training the Untrainable: Introducing Inductive Bias via Representational Alignment
- Blog: Hierarchical Self-supervised Augmented Knowledge Distillation
- Blog: Feature Distillation With Guided Adversarial Contrastive Learning
- Blog: Feature-map-level Online Adversarial Knowledge Distillation
- Blog: Knowledge Distillation for Language Models
- Blog: Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
- Blog: Multi-head Knowledge Distillation for Model Compression
- Blog: Robustness-aware 2-bit quantization with real-time performance for neural network
- Blog: Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification
- Blog: Conditional Generative Data-Free Knowledge Distillation based on Attention Transfer
- Blog: LumiNet: The Bright Side of Perceptual Knowledge Distillation
- Blog: eTag: Class-Incremental Learning via Embedding Distillation and Task-Oriented Generation
- Blog: Self-Attentive Spatio-Temporal Calibration for Precise Intermediate Layer Matching in ANN-to-SNN Distillation
- Blog: FEED: Feature-level Ensemble for Knowledge Distillation
- Blog: MergeNet: Knowledge Migration Across Heterogeneous Models, Tasks, and Modalities
- Blog: KTAN: Knowledge Transfer Adversarial Network
- Blog: Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers
- Blog: Self-Training Based Few-Shot Node Classification by Knowledge Distillation
- Blog: Diving deeper into mentee networks
- Blog: ALP-KD: Attention-Based Layer Projection for Knowledge Distillation
- Blog: Transforming Gaps into Gains: Bridging Model and Data Heterogeneity in Federated Learning via Knowledge Weak-Aware Zones
- Blog: Collaborative Distillation Strategies for Parameter-Efficient Language Model Deployment
- Blog: Learning Without Memorizing
- Blog: Patient Knowledge Distillation for BERT Model Compression
- Blog: Weight Distillation: Transferring the Knowledge in Neural Network Parameters
- Blog: Knowledge Distillation for Enhancing a Lightweight Magnet Tile Target Detection Model: Leveraging Spatial Attention and Multi-Scale Output Features
- Blog: Teach-DETR: Better Training DETR with Teachers
- Blog: CHEER: Rich Model Helps Poor Model via Knowledge Infusion
- Blog: Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
- Blog: Learn From One Specialized Sub-Teacher: One-to-One Mapping for Feature-Based Knowledge Distillation
- Blog: A Progressive Framework of Vision-language Knowledge Distillation and Alignment for Multilingual Scene
- Blog: Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning
- Blog: Knowledge Distillation from Internal Representations
- Blog: Knowledge Distillation via the Target-aware Transformer
Showing 60 of 164 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_single_layer_distillation_bottleneck_e91e2cdd,
title = {single-layer distillation bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/single-layer-distillation-bottleneck/rdr_e91e2cdd2d2f46c082530876ef98f21e},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 2, frozen text):
@misc{lacuna_single_layer_distillation_bottleneck_e91e2cdd_v2,
title = {single-layer distillation bottleneck},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/single-layer-distillation-bottleneck/rdr_e91e2cdd2d2f46c082530876ef98f21e/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}