No article text was recorded for this version.
representation rank collapse
Version 1 of 1136 papers159 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
- Blog: Training Dynamics of Deep Networks using Stochastic Gradient Descent via Neural Tangent Kernel
- Blog: Beyond BatchNorm: Towards a General Understanding of Normalization in Deep Learning
- Blog: Dissecting Deep RL with High Update Ratios: Combatting Value Divergence
- Blog: TF-MAS: Training-free Mamba2 Architecture Search
- Blog: Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning
- Blog: Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit
- Blog: Understanding and Preventing Capacity Loss in Reinforcement Learning
- Blog: Reasoning-preserved Efficient Distillation of Large Language Models via Activation-aware Initialization
- Blog: Avoiding pathologies in very deep networks
- Blog: Extreme Network Compression via Filter Group Approximation
- Blog: Setting the Record Straight on Transformer Oversmoothing
- Blog: Avoiding pathologies in very deep networks
- Blog: Emergence of Globally Attracting Fixed Points in Deep Neural Networks With Nonlinear Activations
- Blog: Persistency of Excitation for Robustness of Neural Networks
- Blog: Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks
- Blog: Initialization of ReLUs for Dynamical Isometry
- Blog: Understanding and Minimising Outlier Features in Neural Network Training
- Blog: SML: Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction
- Blog: On the impact of activation and normalization in obtaining isometric embeddings at initialization
- Blog: Rethinking Channel Dimensions for Efficient Model Design
- Blog: Geometric Dynamics of Signal Propagation Predict Trainability of Transformers
- Blog: Rethinking Oversmoothing in Graph Neural Networks: A Rank-Based Perspective
- Blog: Fast, Convex and Conditioned Network for Multi-Fidelity Vectors and Stiff Univariate Differential Equations
- Blog: Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
- Blog: Loss of plasticity in deep continual learning
- Blog: Hadamax Encoding: Elevating Performance in Model-Free Atari
- Blog: Augmented Shortcuts for Vision Transformers
- Blog: Rank Diminishing in Deep Neural Networks
- Blog: Attention is not all you need: pure attention loses rank doubly exponentially with depth
- Blog: The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning
- Blog: Branch Scaling Manifests as Implicit Architectural Regularization for Improving Generalization in Overparameterized ResNets
- Blog: On the impact of activation and normalization in obtaining isometric embeddings at initialization
- Blog: Singular Value Perturbation and Deep Network Optimization
- Blog: On Bridging the Gap between Mean Field and Finite Width Deep Random Multilayer Perceptron with Batch Normalization
- Blog: Richer priors for infinitely wide multi-layer perceptrons
- Blog: MUSE: Feature Self-Distillation with Mutual Information and Self-Information
- Blog: Transformers Get Stable: An End-to-End Signal Propagation Theory for Language Models
- Blog: On the Origins of the Block Structure Phenomenon in Neural Network Representations
- Blog: Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization
- Blog: Globally Gated Deep Linear Networks
- Blog: Meta Learning Low Rank Covariance Factors for Energy Based Deterministic Uncertainty
- Blog: Simplicial Embeddings Improve Sample Efficiency in Actor–Critic Agents
- Blog: Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
- Blog: Nonlinear random matrix theory for deep learning
- Blog: Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning
- Blog: Initializing ReLU networks in an expressive subspace of weights
- Blog: The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit
- Blog: Parseval Regularization for Continual Reinforcement Learning
- Blog: Parseval Networks: Improving Robustness to Adversarial Examples
- Blog: On the combined effect of class imbalance and concept complexity in deep learning
- Blog: Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
- Blog: Understanding and Preventing Capacity Loss in Reinforcement Learning
- Blog: Stackelberg Coupling of Online Representation Learning and Reinforcement Learning
- Blog: Batch Normalization Orthogonalizes Representations in Deep Random Networks
- Blog: Theoretical Understanding of Batch-normalization: A Markov Chain Perspective
- Blog: Generational PID (G-PID): Real-Time n-Dimensional Control of Stochastic Language Manifolds
- Blog: The Tunnel Effect: Building Data Representations in Deep Neural Networks
- Blog: Connect, Collapse, Corrupt: Learning Cross-Modal Tasks with Uni-Modal Data
- Blog: Superposition in Graph Neural Networks
Showing 60 of 136 papers.
History & citation
Current · Permanent link
- Feature Rank Preservation in Deep Reinforcement Learning merged into this direction (50% overlap)
- Preventing Rank Collapse in Deep Neural Networks merged into this direction (78% overlap)
- Normalization as a Defense Against Rank Collapse merged into this direction (62% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_representation_rank_collapse_c95073dd,
title = {representation rank collapse},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/representation-rank-collapse/rdr_c95073dd2cd547bf8d1e13acdb3fe72d},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_representation_rank_collapse_c95073dd_v1,
title = {representation rank collapse},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/representation-rank-collapse/rdr_c95073dd2cd547bf8d1e13acdb3fe72d/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}