No article text was recorded for this version.
weight magnitude-direction decoupling
Version 1 of 129 papers39 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
- Blog: Low-bit Shift Network for End-to-End Spoken Language Understanding
- Blog: Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
- Blog: On the Benefits of Weight Normalization for Overparameterized Matrix Sensing
- Blog: A new low-rank adaptation method for brain structure and metastasis segmentation via decoupled principal weight direction and magnitude
- Blog: Decoupling Angles and Strength in Low-rank Adaptation
- Blog: DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models
- Blog: Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
- Blog: Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization
- Blog: Implicit Regularization and Convergence for Weight Normalization
- Blog: Implicit Regularization for Group Sparsity
- Blog: Optimization and Generalization Guarantees for Weight Normalization
- Blog: S$^3$: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networks
- Blog: Switchable Whitening for Deep Representation Learning
- Blog: Optimization Theory for ReLU Neural Networks Trained with Normalization Layers
- Blog: Centered Weight Normalization in Accelerating Training of Deep Neural Networks
- Blog: Online Parameter-Free Learning of Multiple Low Variance Tasks
- Blog: Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation
- Blog: A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent
- Blog: Complexity control by gradient descent in deep networks
- Blog: Random Weight Factorization Improves the Training of Continuous Neural Representations
- Blog: Implicit Regularization for Optimal Sparse Recovery
- Blog: Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
- Blog: Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent
- Blog: Analyzing & Eliminating Learning Rate Warmup in GPT Pre-Training
- Blog: Radial and Directional Posteriors for Bayesian Deep Learning
- Blog: Robust implicit regularization via weight normalization
- Blog: Statistical Guarantees for Regularized Neural Networks
- Blog: Path-BN: Towards effective batch normalization in the Path Space for ReLU networks
History & citation
Current · Permanent link
- This direction split from Weight Decomposition into Magnitude and Direction (86% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_weight_magnitude_direction_decoupling_79633e0a,
title = {weight magnitude-direction decoupling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/weight-magnitude-direction-decoupling/rdr_79633e0a9ae84eb69c06f78bc589e9e3},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_weight_magnitude_direction_decoupling_79633e0a_v1,
title = {weight magnitude-direction decoupling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/weight-magnitude-direction-decoupling/rdr_79633e0a9ae84eb69c06f78bc589e9e3/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}