No article text was recorded for this version.
initialization scale sensitivity
Version 1 of 184 papers118 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Algorithmic Regularization in Tensor Optimization: Towards a Lifted Approach in Matrix Sensing
- Blog: Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction
- Blog: Generalization of Two-layer Neural Networks: An Asymptotic Viewpoint
- Blog: A unifying view on implicit bias in training linear neural networks
- Blog: Minimum norm interpolation by perceptra: Explicit regularization and implicit bias
- Blog: Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion
- Blog: How connectivity structure shapes rich and lazy learning in neural circuits
- Blog: Implicit Balancing and Regularization: Generalization and Convergence Guarantees for Overparameterized Asymmetric Matrix Sensing
- Blog: An Analysis for Reasoning Bias of Language Models with Small Initialization
- Blog: Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations
- Blog: Implicit Regularization in Matrix Sensing via Mirror Descent
- Blog: Implicit Regularization in Deep Matrix Factorization
- Blog: Minimum norm interpolation by perceptra: Explicit regularization and implicit bias
- Blog: Implicit Regularization with Polynomial Growth in Deep Tensor Factorization
- Blog: On the spectral bias of two-layer linear networks
- Blog: Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
- Blog: Implicit Regularization in Matrix Factorization
- Blog: Implicit Regularization in Tensor Factorization
- Blog: Precise Dynamics of Diagonal Linear Networks: A Unifying Analysis by Dynamical Mean-Field Theory
- Blog: Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
- Blog: Robust implicit regularization via weight normalization
- Blog: Kernel and Rich Regimes in Overparametrized Models
- Blog: Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry
- Blog: Robust Implicit Regularization via Weight Normalization
- Blog: Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape
- Blog: Deep Linear Networks Dynamics: Low-Rank Biases Induced by Initialization Scale and L2 Regularization
- Blog: Frivolous Units: Wider Networks Are Not Really That Wide
- Blog: The Power of Small Initialization in Noisy Low-Tubal-Rank Tensor Recovery
- Blog: Benignity of loss landscape with weight decay requires both large overparametrization and initialization
- Blog: On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks
- Blog: Implicit Balancing and Regularization: Generalization and Convergence Guarantees for Overparameterized Asymmetric Matrix Sensing
- Blog: Kernel and Rich Regimes in Overparametrized Models
- Blog: Implicit Regularization in Over-Parameterized Support Vector Machine
- Blog: Towards Understanding Learning in Neural Networks with Linear Teachers
- Blog: On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent
- Blog: Implicit Regularization for Group Sparsity
- Blog: The Importance of Positional Encoding Initialization in Transformers for Relational Reasoning
- Blog: Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
- Blog: Towards Understanding Learning in Neural Networks with Linear Teachers
- Blog: Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example
- Blog: Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data
- Blog: Omnigrok: Grokking Beyond Algorithmic Data
- Blog: Early alignment in two-layer networks training is a two-edged sword
- Blog: Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization
- Blog: The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features
- Blog: On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks
- Blog: A Quantitative Approach to Predicting Representational Learning and Performance in Neural Networks
- Blog: Memorization in Overparameterized Autoencoders
- Blog: Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry
- Blog: Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations
- Blog: Humans and neural networks show similar patterns of transfer and interference during continual learning
- Blog: How to induce regularization in generalized linear models: A guide to reparametrizing gradient flow
- Blog: Implicit Regularization in Matrix Factorization
- Blog: On Implicit Bias in Overparameterized Bilevel Optimization
- Blog: LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won't Fail)
- Blog: Transformers learn through gradual rank increase
- Blog: On the Minimal Degree Bias in Generalization on the Unseen for non-Boolean Functions
- Blog: Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs
- Blog: Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning
- Blog: Implicit Regularization in Deep Tucker Factorization: Low-Rankness via Structured Sparsity
Showing 60 of 84 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_initialization_scale_sensitivity_02a386d0,
title = {initialization scale sensitivity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/initialization-scale-sensitivity/rdr_02a386d019054af1b5aab45a25fc2ef4},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_initialization_scale_sensitivity_02a386d0_v1,
title = {initialization scale sensitivity},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/initialization-scale-sensitivity/rdr_02a386d019054af1b5aab45a25fc2ef4/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}