No article text was recorded for this version.
feed-forward network overhead
Version 1 of 127 papers31 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Of Non-Linearity and Commutativity in BERT
- Blog: WK, WV is (Linearly) All You Need: On the Necessity of the QKV Weight Triplet in Self-Attention Transformers
- Blog: Speech and Language Recognition with Low-rank Adaptation of Pretrained Models
- Blog: DaViT: Dual Attention Vision Transformers
- Blog: MemoryLLM: Plug-n-Play Interpretable Feed-Forward Memory for Transformers
- Blog: Attention Mechanism with Energy-Friendly Operations
- Blog: InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
- Blog: CoLT5: Faster Long-Range Transformers with Conditional Computation
- Blog: EfficientASR: Speech Recognition Network Compression via Attention Redundancy and Chunk-Level FFN Optimization
- Blog: MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
- Blog: GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
- Blog: SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
- Blog: MSCFFN: A New FFN with Multi-Space Cross to Accelerate Transformer
- Blog: Hierarchical Transformers Are More Efficient Language Models
- Blog: Deeply Tensor Compressed Transformers for End-to-End Object Detection
- Blog: PSLT: A Light-Weight Vision Transformer With Ladder Self-Attention and Progressive Shift
- Blog: Building on Efficient Foundations: Effective Training of LLMs with Structured Feedforward Layers
- Blog: ToaSt: Token Channel Selection and Structured Pruning for Efficient ViT
- Blog: Efficient Fine-Tuning of BERT Models on the Edge
- Blog: Rethinking the Self-Attention in Vision Transformers
- Blog: Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
- Blog: NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning
- Blog: PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
- Blog: PartialFormer: Modeling Part Instead of Whole for Machine Translation
- Blog: Sparser, Faster, Lighter Transformer Language Models
- Blog: Enhancing Vision Transformer: Amplifying Non-Linearity in Feedforward Network Module
- Blog: RePaViT: Scalable Vision Transformer Acceleration via Structural Reparameterization on Feedforward Network Layers
History & citation
Current · Permanent link
- This direction split from Sparsifying the Feed-Forward Bottleneck in Transformers (85% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_feed_forward_network_overhead_780a6f91,
title = {feed-forward network overhead},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/feed-forward-network-overhead/rdr_780a6f91f5f14f1cad6b961f418a3bad},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_feed_forward_network_overhead_780a6f91_v1,
title = {feed-forward network overhead},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/feed-forward-network-overhead/rdr_780a6f91f5f14f1cad6b961f418a3bad/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}