No article text was recorded for this version.
distillation mechanism ambiguity
Version 2 of 2107 papers116 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation
- Blog: Knowledge Distillation: The Functional Perspective
- Blog: Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
- Blog: KDExplainer: A Task-oriented Attention Model for Explaining Knowledge Distillation
- Blog: The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model
- Blog: Born-Again Neural Networks
- Blog: Knowledge Distillation: A Survey
- Blog: Born Again Neural Networks
- Blog: Extractive Knowledge Distillation
- Blog: Regression as a Simple Yet Effective Tool for Self-supervised Knowledge Distillation
- Blog: Aligning Large and Small Language Models via Chain-of-Thought Reasoning
- Blog: Toward Understanding Privileged Features Distillation in Learning-to-Rank
- Blog: No Head Left Behind - Multi-Head Alignment Distillation for Transformers
- Blog: A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions
- Blog: SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines
- Blog: Quantification of Large Language Model Distillation
- Blog: A Study of Non-autoregressive Model for Sequence Generation
- Blog: Knowledge Distillation Performs Partial Variance Reduction
- Blog: StochCA: A novel approach for exploiting pretrained models with cross-attention
- Blog: Augmenting interpretable models with large language models during training
- Blog: Towards a theory of model distillation
- Blog: Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
- Blog: Sparse Logits Suffice to Fail Knowledge Distillation
- Blog: Toward Understanding Privileged Features Distillation in Learning-to-Rank
- Blog: Knowledge Distillation ≈ Label Smoothing: Fact or Fallacy?
- Blog: Bayesian Knowledge Distillation: A Bayesian Perspective of Distillation with Uncertainty Quantification
- Blog: Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scaling
- Blog: Knowledge Distillation Beyond Model Compression
- Blog: Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation
- Blog: Knowledge Distillation on Graphs: A Survey
- Blog: Distilling Text Style Transfer With Self-Explanation From LLMs
- Blog: Investigating Mysteries of CoT-Augmented Distillation
- Blog: Efficiently Robustify Pre-trained Models
- Blog: Towards Understanding Knowledge Distillation
- Blog: Quantifying Knowledge Distillation using Partial Information Decomposition
- Blog: A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
- Blog: On Explaining Knowledge Distillation: Measuring and Visualising the Knowledge Transfer Process
- Blog: Explaining Knowledge Distillation by Quantifying the Knowledge
- Blog: A Theoretical Study of Dataset Distillation
- Blog: Distilling Ensembles Improves Uncertainty Estimates
- Blog: Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks
- Blog: Towards Understanding Distilled Reasoning Models: A Representational Approach
- Blog: Generative Prompt Internalization
- Blog: Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective
- Blog: Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
- Blog: Self-knowledge distillation via dropout
- Blog: Unified Text and Speaker Verification using SSL model for Text-Dependent Speaker Verification
- Blog: Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield
- Blog: CAKD: A Correlation-Aware Knowledge Distillation Framework Based on Decoupling Kullback-Leibler Divergence
- Blog: How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives
- Blog: Improved Knowledge Distillation via Teacher Assistant
- Blog: Class Attention Transfer Based Knowledge Distillation
- Blog: On the Impact of Knowledge Distillation for Model Interpretability
- Blog: Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification
- Blog: Explaining Knowledge Distillation by Quantifying the Knowledge
- Blog: Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer
- Blog: Task-agnostic Distillation of Encoder-Decoder Language Models
- Blog: Interpretable Embedding Procedure Knowledge Transfer via Stacked Principal Component Analysis and Graph Neural Network
- Blog: Is Ockham’s razor losing its edge? New perspectives on the principle of model parsimony
- Blog: IDEAL: Query-Efficient Data-Free Learning from Black-Box Models
Showing 60 of 107 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_distillation_mechanism_ambiguity_9b37eb9b,
title = {distillation mechanism ambiguity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/distillation-mechanism-ambiguity/rdr_9b37eb9b2fd24010835058c78a5084dd},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_distillation_mechanism_ambiguity_9b37eb9b_v2,
title = {distillation mechanism ambiguity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/distillation-mechanism-ambiguity/rdr_9b37eb9b2fd24010835058c78a5084dd/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}