No article text was recorded for this version.
compute-capacity coupling
Version 1 of 166 papers66 conceptsfrozen 2026-10-04
Papers in this version
- Blog: 3M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognition
- Blog: Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers
- Blog: Inner-layer Token Self-modulation as Another Scaling Axis for LLMs
- Blog: Lifting the Curse of Capacity Gap in Distilling Language Models
- Blog: Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models
- Blog: MIDUS: Memory-Infused Depth Up-Scaling
- Blog: Rail-only: A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters
- Blog: Learning Factored Representations in a Deep Mixture of Experts
- Blog: Mixture of Experts (MoE): A Big Data Perspective
- Blog: Mixture of Attention Heads: Selecting Attention Heads Per Token
- Blog: CLIP-FMoE: Scalable CLIP via Fused Mixture-of-Experts with Enforced Specialization
- Blog: cMoLLM at Scale: Horizontal Scaling Laws for Convolutionally-Gated Mixture-of-LLMs
- Blog: Lookup-Table Recurrent Language Models for Long Tail Speech Recognition
- Blog: EWMoE: An Effective Model for Global Weather Forecasting with Mixture-of-Experts
- Blog: GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- Blog: Smallcap: Lightweight Image Captioning Prompted with Retrieval Augmentation
- Blog: Improving Language Models by Retrieving from Trillions of Tokens
- Blog: STEM: SCALING TRANSFORMERS WITH EMBEDDING MODULES
- Blog: LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training
- Blog: Fanar 2.0: Arabic Generative AI Stack
- Blog: GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- Blog: Ares: Loss-Free Mixture-of-Experts Routing for Bidirectional Protein Encoders
- Blog: Memory Layers at Scale
- Blog: CondConv: Conditionally Parameterized Convolutions for Efficient Inference
- Blog: Large Memory Layers with Product Keys
- Blog: Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasoners
- Blog: GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
- Blog: Mixture of Attention Heads: Selecting Attention Heads Per Token
- Blog: MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts
- Blog: GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Blog: DataMUX: Data Multiplexing for Neural Networks
- Blog: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Blog: Scaling Vision with Sparse Mixture of Experts
- Blog: Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
- Blog: LiRank: Industrial Large Scale Ranking Models at LinkedIn
- Blog: SparseEHR: Scalable Foundation Modeling for Structured EHR via Conditional Computation
- Blog: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- Blog: On the effectiveness of discrete representations in sparse mixture of experts
- Blog: BASE Layers: Simplifying Training of Large, Sparse Models
- Blog: Parameters vs FLOPs: Scaling Laws for Optimal Sparsity for Mixture-of-Experts Language Models
- Blog: Does Preprocessing Help Training Over-parameterized Neural Networks?
- Blog: Mixture of Hidden-Dimensions: Not All Hidden-States’ Dimensions are Needed in Transformer
- Blog: Gradient Transformer: Learning to Generate Updates for LLMs
- Blog: DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- Blog: Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning
- Blog: EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing
- Blog: Towards Distributed Neural Architectures
- Blog: StableMoE: Stable Routing Strategy for Mixture of Experts
- Blog: Implicit Neural Representations with Levels-of-Experts
- Blog: Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers
- Blog: CR-MoE: Consistent Routed Mixture-of-Experts for Scaling Contrastive Learning
- Blog: Residual Matrix Transformers: Scaling the Size of the Residual Stream
- Blog: Efficient Large Language Models: A Survey
- Blog: BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts
- Blog: Scaling Diffusion Transformers to 16 Billion Parameters
- Blog: GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- Blog: Mixture of Experts Enable Efficient and Effective Protein Understanding and Design
- Blog: Investigation of Scaling Laws for Encoder-Decoder Protein Language Models
- Blog: A Survey on Efficient Training of Transformers
- Blog: DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
Showing 60 of 66 papers.
History & citation
Current · Permanent link
- This direction split from Sparse Mixture of Experts for Efficient Model Scaling (74% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_compute_capacity_coupling_9be82b88,
title = {compute-capacity coupling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/compute-capacity-coupling/rdr_9be82b88b57c4ff78764111c8ffe87dd},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 1, frozen text):
@misc{lacuna_compute_capacity_coupling_9be82b88_v1,
title = {compute-capacity coupling},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/compute-capacity-coupling/rdr_9be82b88b57c4ff78764111c8ffe87dd/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}