When a large language model (LLM) is trained, it learns a massive, generalized representation of language distributed across billions of parameters. However, deploying these foundation models for specialized downstream tasks—such as text-to-SQL generation, medical diagnosis, or recommendation systems—typically requires adaptation.
The naive approach is full fine-tuning, where every weight in the network is updated via backpropagation. While effective, full fine-tuning introduces a massive computational bottleneck. It requires storing not only the model weights but also optimizer states (such as the first and second moments in Adam), gradients, and intermediate layer activations for every single parameter. This memory overhead can easily exceed the capacity of standard hardware, making adaptation prohibitively expensive for most practitioners.
To bypass this wall, researchers have shifted toward parameter-efficient and training-free adaptation strategies. These methods work because task-specific adaptation does not require rebuilding the model's core internal representations; instead, it requires steering or modulating them.
Subspace and Frequency Projections
The fundamental intuition behind Parameter-Efficient Fine-Tuning (PEFT) is that the weight updates ($\Delta W$) necessary for a model to learn a new task inhabit a subspace with a much lower intrinsic dimensionality than the full weight matrix. Instead of updating a high-dimensional matrix directly, we can optimize a compressed representation of the updates.
In Minimal yet Mighty Adaptation via Hierarchical Cosine Projection, Yixian Shen et al. take this concept into the frequency domain using the Discrete Cosine Transform (DCT). Their framework, MaCP, projects weight updates into spatial frequencies and selectively retains only the critical, high-energy components.
Why does this work? In natural language processing, the structural modifications required to alter a model's behavior are often smooth and low-frequency in nature. By fine-tuning only these essential frequency coefficients and discarding redundant high-frequency spatial updates, the authors drastically reduce both the number of trainable parameters and the memory footprint while maintaining high downstream performance.
Exploiting Structural and Activation Sparsity
Another way to mitigate computational costs is to recognize that large language models naturally exhibit high degrees of activation sparsity. For any given token or task, a significant portion of the network's neurons contribute negligibly to the final output. Standard training loops waste immense compute by calculating gradients for these inactive pathways.
To exploit this phenomenon, Da Ma et al. introduced Sparsity-Accelerated Training (SAT) in Sparsity-Accelerated Training for Large Language Models. SAT dynamically identifies and skips computations for inactive neurons during secondary training phases like supervised fine-tuning. Because the underlying model architecture naturally routes information through sparse sub-networks, skipping these redundant calculations yields up to a 38% reduction in training time without sacrificing the model's final accuracy.
Eliminating Backpropagation via Zeroth-Order Optimization
The primary culprit behind GPU memory exhaustion during fine-tuning is backpropagation itself. To compute gradients, the system must perform a forward pass and cache the intermediate activations of every layer in memory until the backward pass completes. For long sequences and massive models, this activation memory dwarfs the memory required to store the model weights.
Zeroth-Order (ZO) optimization bypasses backpropagation entirely by estimating gradients using only forward passes. By applying small, random perturbations to the weights and observing the change in the loss function, ZO methods can optimize models without ever caching activations or computing analytical gradients.
However, standard ZO optimization suffers from high variance and slow convergence because random perturbations rarely align perfectly with the true descent direction. To resolve this, Zhendong Mi et al. proposed KerZOO in KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning. The authors incorporate kernel functions to exploit historical optimization trajectories, significantly reducing gradient estimation bias. This allows KerZOO to accelerate convergence, making gradient-free, memory-restricted LLM fine-tuning practically viable.
Training-Free Weight Modulation
The most computationally efficient fine-tuning strategy is one that requires no training at all. Recent work suggests that the latent capabilities required for specialized tasks are already present within pre-trained foundation models, but are occasionally obscured by structural noise or misaligned weight distributions.
In Unleashing the Potential of Large Language Models through Spectral Modulation, Peng Sun et al. treat an LLM's pre-trained weight matrices as 2D signals. Instead of using gradient descent to alter the weights, they apply a post-processing technique called "spectral modulation" to denoise the weight matrices directly in the frequency domain. By filtering out performance-constraining noise while preserving the core mathematical logic of the model, they boost zero-shot accuracy entirely through mathematical manipulation of the static weights.
Similarly, transferring specialized behaviors across different model architectures typically requires starting the fine-tuning process from scratch on the new architecture. To avoid this, Barry Wang et al. developed Command-V in Command-V: Training-Free Representation Finetuning Transfer. Their approach allows alignment behaviors (such as safety filters or specific reasoning styles) to be transferred directly between different model architectures without undergoing intensive backpropagation-based fine-tuning, demonstrating that representation spaces can be aligned and mapped without explicit retraining.