Module-wise Adaptive Distillation for Multimodality Foundation Models

Chen Liang, Jiahui Yu, Ming-Hsuan Yang, Matthew Brown, Yin Cui, Tuo Zhao, Boqing Gong, Tianyi Zhou

Introduction

Large-scale multimodal foundation models, such as CoCa and CLIP, have set new benchmarks across various vision-language tasks. However, their massive parameter counts make them computationally prohibitive for real-time deployment. Knowledge distillation, specifically layerwise distillation (LWD), is a common technique to compress these "teacher" models into smaller "student" models.

Traditional LWD approaches typically treat all architectural modules—such as image encoders, text encoders, and multimodal decoders—with equal importance throughout the training process. This research identifies a significant inefficiency in this uniform approach: different tasks rely on different modules, and distilling irrelevant components can lead to gradient interference, hindering the student's learning efficiency.

The OPTIMA Framework

To address these limitations, the authors introduce OPTIMA (Module-wise Adaptive Distillation with Multi-arm Bandit). OPTIMA treats the selection of modules for distillation as a non-stationary Multi-Armed Bandit (MAB) problem.

In this framework, each "arm" represents a specific module or a combination of modules. The "reward" for selecting an arm is defined by the reduction in distillation loss. Because the relative importance of modules shifts as the student model matures, OPTIMA employs a modified Thompson Sampling algorithm integrated with an Exponential Moving Average (EMA). This allows the system to adaptively prioritize modules that offer the highest performance gains at any given stage of training.

The OPTIMA framework utilizes a Multi-armed Bandit algorithm to adaptively select which modules to distill in each training round.

Task-Specific Dynamics and Interference

A key insight of this work is that module importance is highly task-dependent. For instance, in image captioning tasks (e.g., COCO Caption), the image encoder is the primary driver of performance. Conversely, for reasoning tasks like NLVR2, the multimodal decoder plays a more critical role.

Standard distillation methods that apply uniform weights across all modules often underperform compared to targeted distillation. By adaptively selecting the most relevant modules, OPTIMA avoids the "interference" caused by training on less critical components, leading to more robust student models.

Comparison of task performance when distilling different modules, highlighting that module importance varies significantly between NLVR2 and COCO Captioning.

Experimental Results

The researchers evaluated OPTIMA by distilling a CoCa-Large teacher (672M parameters) into CoCa-Tiny students (55M and 102M parameters). The benchmarks included VQA v2.0, SNLI-VE, NLVR2, and Microsoft COCO Caption.

  • Performance Gains: OPTIMA consistently outperformed standard LWD baselines. On the COCO Captioning benchmark, OPTIMA achieved a CIDEr score of 121.2, compared to 116.8 for standard LWD.
  • Model Efficiency: Remarkably, a 102M-parameter student trained via OPTIMA outperformed the 293M-parameter CoCa-Base model on three out of four benchmarks, demonstrating that adaptive distillation can produce highly efficient models that punch above their weight class.
  • Generalization: Student models trained with OPTIMA showed lower prediction loss and improved output discrepancy scores, indicating superior generalization to unseen data compared to fixed-distillation strategies.

Conclusion and Future Directions

OPTIMA demonstrates that adaptive, module-wise distillation is a powerful strategy for compressing multimodal foundation models. By framing module selection as a bandit problem, the framework navigates the complex trade-offs between different modalities and tasks.

Future work may explore finer-grained module selection (e.g., specific layer groups) and investigate the dependencies between coupled modules using more advanced reinforcement learning techniques. While currently focused on vision-language models, the OPTIMA framework is potentially applicable to a wide range of multimodal architectures, including those involving audio and video.

Summary updated

Abstract

Pre-trained multimodal foundation models have demonstrated remarkable generalizability but pose challenges for deployment due to their large sizes. One effective approach to reducing their sizes is layerwise distillation, wherein small student models are trained to match the hidden representations of large teacher models at each layer. Motivated by our observation that certain architecture components, referred to as modules, contribute more significantly to the student's performance than others, we propose to track the contributions of individual modules by recording the loss decrement after distillation each module and choose the module with a greater contribution to distill more frequently. Such an approach can be naturally formulated as a multi-armed bandit (MAB) problem, where modules and loss decrements are considered as arms and rewards, respectively. We then develop a modified-Thompson sampling algorithm named OPTIMA to address the nonstationarity of module contributions resulting from model updating. Specifically, we leverage the observed contributions in recent history to estimate the changing contribution of each module and select modules based on these estimations to maximize the cumulative contribution. We evaluate the effectiveness of OPTIMA through distillation experiments on various multimodal understanding and image captioning tasks, using the CoCa-Large model \citep{yu2022coca} as the teacher model.
BibTeX
@inproceedings{liang2023module,
  author = {Chen Liang and Jiahui Yu and Ming-Hsuan Yang and Matthew Brown and Yin Cui and Tuo Zhao and Boqing Gong and Tianyi Zhou},
  title = {{Module-wise Adaptive Distillation for Multimodality Foundation Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=JhQP33aMx2},
}
Figure