No article text was recorded for this version.
non-linear activation gap
Version 1 of 1342 papers349 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective
- Blog: On the Learning Dynamics of Two-layer Linear Networks with Label Noise SGD
- Blog: Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
- Blog: A note on Linear Bottleneck networks and their Transition to Multilinearity
- Blog: Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking)
- Blog: An Overview of Low-Rank Structures in the Training and Adaptation of Large Models
- Blog: The Hessian perspective into the Nature of Convolutional Neural Networks
- Blog: On Non-local Convergence Analysis of Deep Linear Networks
- Blog: On Computational Entanglement and Its Interpretation in Adversarial Machine Learning
- Blog: Understanding the Dynamics of Gradient Flow in Overparameterized Linear models
- Blog: On the Convergence of Gradient Flow on Multi-layer Linear Models
- Blog: Can Stability be Detrimental? Better Generalization through Gradient Descent Instabilities
- Blog: PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
- Blog: Targeted collapse regularized autoencoder for anomaly detection: black hole at the center
- Blog: Three Mechanisms of Feature Learning in a Linear Network
- Blog: The Implicit Bias of Depth: How Incremental Learning Drives Generalization
- Blog: Sliced Recurrent Neural Networks
- Blog: Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
- Blog: Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks
- Blog: From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
- Blog: On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
- Blog: Implicit Regularization via Spectral Neural Networks and Non-linear Matrix Sensing
- Blog: Implicit Regularization in Deep Matrix Factorization
- Blog: Energy Preservation and Stability of Random Filterbanks
- Blog: Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes
- Blog: Order parameters and phase transitions of continual learning in deep neural networks
- Blog: A Complete Loss Landscape Analysis of Regularized Deep Matrix Factorization
- Blog: The Geometry of the Set of Equivalent Linear Neural Networks
- Blog: On the Mathematical Understanding of ResNet with Feynman Path Integral
- Blog: What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models
- Blog: On the Impact of Downstream Tasks on Sampling and Reconstructing Noisy Graph Signals
- Blog: Generalized Tensor Models for Recurrent Neural Networks
- Blog: Feature Expansion for Graph Neural Networks
- Blog: The Hessian perspective into the Nature of Convolutional Neural Networks
- Blog: A representer theorem for deep neural networks
- Blog: Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model
- Blog: RADE: Random Add-Drop Edge as a Regularizer
- Blog: Sharp Bounds for Poly-GNNs and the Effect of Graph Noise
- Blog: Feature Collapse
- Blog: Parallel Layer Normalization for Universal Approximation
- Blog: Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks
- Blog: On the Expressivity of Selective State-Space Layers: A Multivariate Polynomial Approach
- Blog: What shapes the loss landscape of self-supervised learning?
- Blog: On the Structural Sensitivity of Deep Convolutional Networks to the Directions of Fourier Basis Functions
- Blog: The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks
- Blog: Implicit Regularization in Deep Matrix Factorization
- Blog: Defects of Convolutional Decoder Networks in Frequency Representation
- Blog: Sharpness of Minima in Deep Matrix Factorization: Exact Expressions
- Blog: Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization
- Blog: Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation
- Blog: Singular Value Representation: A New Graph Perspective On Neural Networks
- Blog: Representation Learning Dynamics of Self-Supervised Models
- Blog: Convergence Analysis of Over-parameterized Deep Linear Networks, and the Principal Components Bias
- Blog: Oscillations Make Neural Networks Robust to Quantization
- Blog: Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth
- Blog: A unifying view on implicit bias in training linear neural networks
- Blog: Optimisation & Generalisation in Networks of Neurons
- Blog: A Local Polyak-Łojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models
- Blog: Short-Term Memory Optimization in Recurrent Neural Networks by Autoencoder-based Initialization
- Blog: Fokker-Planck to Callan-Symanzik: evolution of weight matrices under training
Showing 60 of 342 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_non_linear_activation_gap_3a0583c6,
title = {non-linear activation gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/non-linear-activation-gap/rdr_3a0583c64f4b4f14a3e30b1002e20dba},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_non_linear_activation_gap_3a0583c6_v1,
title = {non-linear activation gap},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/non-linear-activation-gap/rdr_3a0583c64f4b4f14a3e30b1002e20dba/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}