No article text was recorded for this version.
stochastic symmetry breaking (symmetry-breaking initialization)
Version 1 of 164 papers103 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization?
- Blog: Symmetry-Breaking Convergence Analysis of Certain Two-layered Neural Networks with ReLU nonlinearity
- Blog: SPARKLING: Balancing Signal Preservation and Symmetry Breaking for Width-Progressive Learning
- Blog: Learning dynamics of deep linear networks with multiple pathways
- Blog: IDInit: A Universal and Stable Initialization Method for Neural Network Training
- Blog: Analyzing Stream Collapse in Hyper-Connections: From Diagnosis to Mitigation
- Blog: Kernel Stein Discrepancy Descent
- Blog: LEMON: Lossless model expansion
- Blog: The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
- Blog: Reducing Neural Network Parameter Initialization Into an SMT Problem (Student Abstract)
- Blog: Globally Optimal On-line Learning Rules
- Blog: Convergence Analysis of Two-layer Neural Networks with ReLU Activation
- Blog: Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
- Blog: $μ$pscaling small models: Principled warm starts and hyperparameter transfer
- Blog: Transformers Provably Learn Two-Mixture of Linear Classification via Gradient Flow
- Blog: bert2BERT: Towards Reusable Pretrained Language Models
- Blog: Single-Step Initialization for Exploratory Parallel Rollouts in Diffusion LLMs
- Blog: Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
- Blog: Kernel Stein Discrepancy Descent
- Blog: Remove Symmetries to Control Model Expressivity and Improve Optimization
- Blog: IDInit: A Universal and Stable Initialization Method for Neural Network Training
- Blog: Practical recommendations for gradient-based training of deep architectures
- Blog: Training stochastic stabilized supralinear networks by dynamics-neutral growth
- Blog: Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
- Blog: Non-asymptotic convergence bounds for Wasserstein approximation using point clouds
- Blog: Towards a More Complete Theory of Function Preserving Transforms
- Blog: Sinusoidal Initialization, Time for a New Start
- Blog: An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis
- Blog: Sinusoidal Initialization, Time for a New Start
- Blog: Incremental pre-training from smaller language models
- Blog: Flat Channels to Infinity in Neural Loss Landscapes
- Blog: How to Train Your MAML to Excel in Few-Shot Classification
- Blog: ELLE: Efficient Lifelong Pre-training for Emerging Data
- Blog: Leader Stochastic Gradient Descent for Distributed Training of Deep Learning Models
- Blog: AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
- Blog: BeamLoRA: Beam-Constraint Low-Rank Adaptation
- Blog: Automated Search for Resource-Efficient Branched Multi-Task Networks
- Blog: Initial Guessing Bias: How Untrained Networks Favor Some Classes
- Blog: Deep learning via message passing algorithms based on belief propagation
- Blog: BeamLoRA: Beam-Constraint Low-Rank Adaptation
- Blog: On Variational Inference in Biclustering Models
- Blog: Adaptive Neural Networks Using Residual Fitting
- Blog: 2x Faster Language Model Pre-training via Masked Structural Growth
- Blog: Characterizing Evolution in Expectation-Maximization Estimates for Overspecified Mixed Linear Regression
- Blog: Geometric Clustering Using the Information Bottleneck Method
- Blog: Balancing Symmetry and Efficiency in Graph Flow Matching
- Blog: The Learning Dynamcis of a Universal Approximator
- Blog: ELLE: Efficient Lifelong Pre-training for Emerging Data
- Blog: Net2Net: Accelerating Learning via Knowledge Transfer Tianqi Chen
- Blog: A Multi-Level Framework for Accelerating Training Transformer Models
- Blog: Chaitin-Kolmogorov Complexity and Generalization in Neural Networks
- Blog: Structured Partial Stochasticity in Bayesian Neural Networks
- Blog: Symmetry Considerations for Learning Task Symmetric Robot Policies
- Blog: Convolution Aware Initialization
- Blog: Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
- Blog: A Multi-Level Framework for Accelerating Training Transformer Models
- Blog: DiffeoMorph: Learning to Morph 3D Shapes Using Differentiable Agent-Based Simulations
- Blog: Notes on the Symmetries of 2-Layer ReLU-Networks
- Blog: Symmetry Induces Structure and Constraint of Learning
- Blog: The Symmetry Trap: Parametric Equilibria and the Welfare Cost of Architectural Monoculture
Showing 60 of 64 papers.
History & citation
Current · Permanent link
- Symmetry Breaking in Neural Network Optimization merged into this direction (75% overlap)
- Efficient Model Expansion through Progressive Growth merged into this direction (58% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_stochastic_symmetry_breaking_symmetry_br_006daf2e,
title = {stochastic symmetry breaking (symmetry-breaking initialization)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/stochastic-symmetry-breaking-symmetry-breaking-initialization/rdr_006daf2e489d4f04bafc67d8eede9219},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_stochastic_symmetry_breaking_symmetry_br_006daf2e_v1,
title = {stochastic symmetry breaking (symmetry-breaking initialization)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/stochastic-symmetry-breaking-symmetry-breaking-initialization/rdr_006daf2e489d4f04bafc67d8eede9219/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}