TRAM: Bridging Trust Regions and Sharpness Aware Minimization

Tom Sherborne, Naomi Saphra, Pradeep Dasigi, Hao Peng

Introduction

Fine-tuning massive pre-trained models on downstream tasks is a cornerstone of modern machine learning. However, this process often suffers from "catastrophic forgetting," where the model discards general, robust features learned during pre-training to over-fit the specific nuances of a smaller dataset. To address this, researchers have traditionally turned to two distinct strategies: Sharpness-Aware Minimization (SAM), which seeks flat minima in the parameter space, and Trust Region (TR) methods, which regularize the model's functional representations.

The TRAM Framework

Trust Region Aware Minimization (TRAM) bridges these two approaches. While standard SAM uses a fixed-radius adversarial neighborhood ($\rho$) to find flat regions in the loss landscape, it does not explicitly account for the preservation of pre-trained representation structures. TRAM replaces this fixed neighborhood with a dynamic trust region bound ($d$) that is sensitive to changes in the model's internal representations.

Conceptual overview of TRAM bridging function space trust regions and parameter space SAM ascent.

By tying the SAM perturbation to a trust region metric—such as KL divergence from the pre-trained model ($\theta_0$) or the previous training step ($\theta_{t-1}$)—TRAM ensures that the "sharpness-seeking" optimization stays within a region where the model's functional behavior remains stable. This allows the optimizer to find solutions that are both parameter-wise flat and representation-wise smooth.

Experimental Evaluation

The authors evaluated TRAM across vision, cross-domain language modeling, and cross-lingual transfer tasks. In vision tasks using ViT-base models, TRAM consistently outperformed baselines like ASAM and FSAM. For instance, on the Oxford Flowers dataset, TRAM reached 97.07% accuracy, significantly higher than FSAM's 95.26%.

Performance of TRAM variants compared to baselines on cross-domain language modeling tasks (M2D2 corpus).

In cross-domain language modeling (M2D2 corpus), TRAM demonstrated exceptional robustness. As shown in the figure above, TRAM variants (stars) consistently achieve lower perplexity on out-of-distribution (OOD) tasks compared to standard optimizers like Adam or SAM. Notably, in "hard" transfer scenarios where training and test performance are anticorrelated (e.g., Art and Philosophy domains), TRAM effectively "bucks the trend," maintaining pre-trained generality while learning new domain-specific information.

Key Insights and Conclusion

TRAM's success stems from its ability to maintain high representation similarity to the pre-trained model, as verified by Centered Kernel Alignment (CKA) analysis. By making the optimizer "aware" of representation curvature, TRAM achieves lower out-of-domain sharpness than SAM itself.

While TRAM introduces some computational overhead due to additional forward passes, variants like TRAM-Fisher offer a more efficient alternative. Future work may explore different distance metrics beyond KL divergence to further refine the trust region boundaries. Ultimately, TRAM provides a powerful new tool for robust fine-tuning, ensuring that the wealth of knowledge in pre-trained models is preserved and effectively adapted.

Summary updated

BibTeX
@inproceedings{sherborne2024tram,
  author = {Tom Sherborne and Naomi Saphra and Pradeep Dasigi and Hao Peng},
  title = {{TRAM: Bridging Trust Regions and Sharpness Aware Minimization}},
  booktitle = {International Conference on Learning Representations},
  year = {2024},
  url = {https://openreview.net/forum?id=kxebDHZ7b7},
}
Figure