No article text was recorded for this version.
heavy-tailed self-regularization
Version 2 of 274 papers150 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Random matrix theory analysis of neural network weight matrices
- Blog: Traditional and Heavy Tailed Self Regularization in Neural Network Models
- Blog: Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data
- Blog: Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks
- Blog: Traditional and Heavy-Tailed Self Regularization in Neural Network Models
- Blog: Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
- Blog: Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
- Blog: Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data
- Blog: Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
- Blog: Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping
- Blog: Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning
- Blog: Random matrix analysis of deep neural network weight matrices
- Blog: AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
- Blog: Grokking and Generalization Collapse: Insights from HTSR theory
- Blog: AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
- Blog: Inductive Bias and Spectral Properties of Single-Head Attention in High Dimensions
- Blog: Model Balancing Helps Low-data Training and Fine-tuning
- Blog: Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
- Blog: Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data
- Blog: Model Balancing Helps Low-data Training and Fine-tuning
- Blog: Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics
- Blog: From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
- Blog: Models of Heavy-Tailed Mechanistic Universality
- Blog: From SGD to Spectra: A Theory of Neural Network Weight Dynamics
- Blog: AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
- Blog: Transferable and Principled Efficiency for Open-Vocabulary Segmentation
- Blog: Models of Heavy-Tailed Mechanistic Universality
- Blog: AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
- Blog: Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training
- Blog: The SVD of Convolutional Weights: A CNN Interpretability Framework
- Blog: Interpreting Deep Graph Convolutional Networks with Spectrum Perspective
- Blog: Spectral Signatures of Large Language Models
- Blog: Locating Information in Large Language Models via Random Matrix Theory
- Blog: A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities
- Blog: Grokking, Rank Minimization and Generalization in Deep Learning
- Blog: Implicit Data-Driven Regularization in Deep Neural Networks under SGD
- Blog: Boundary between noise and information applied to filtering neural network weight matrices
- Blog: Spectral Evolution and Invariance in Linear-width Neural Networks
- Blog: Weight matrices compression based on PDB model in deep neural networks
- Blog: AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
- Blog: Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
- Blog: AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
- Blog: One LR Doesn’t Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs
- Blog: Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws
- Blog: From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
- Blog: Grokking and Generalization Collapse: Insights from HTSR theory
- Blog: Emergent Specialization: Rare Token Neurons in Language Models
- Blog: Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning
- Blog: Using Pre-trained LLMs for Multivariate Time Series Forecasting
- Blog: On Bridging the Gap between Mean Field and Finite Width Deep Random Multilayer Perceptron with Batch Normalization
- Blog: The recent Physics and Chemistry Nobel Prizes, AI, and the convergence of knowledge fields
- Blog: Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks
- Blog: SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
- Blog: On the Spectral Differences Between NTK and CNTK and Their Implications for Point Cloud Recognition
- Blog: AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality
- Blog: Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning
- Blog: Transferable and Principled Efficiency for Open-Vocabulary Segmentation
- Blog: EigenShield: Inference-Time, Model-Agnostic Jailbreaking Defense via Causal Subspace Filtering
- Blog: FedEMoE: Improving Personalization on Heterogeneous Federated Learning via Elastic Mixture of Experts Architecture
- Blog: FlexLoRA: Entropy-Guided Flexible Low-Rank Adaptation
Showing 60 of 74 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_heavy_tailed_self_regularization_469c9b32,
title = {heavy-tailed self-regularization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/heavy-tailed-self-regularization/rdr_469c9b32356147d9837e048aaa34576d},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 2, frozen text):
@misc{lacuna_heavy_tailed_self_regularization_469c9b32_v2,
title = {heavy-tailed self-regularization},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/heavy-tailed-self-regularization/rdr_469c9b32356147d9837e048aaa34576d/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}