Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Space

Gaurav Verma, Minje Choi, Kartik Sharma, Jamelle Watson-Daniels, Sejoon Oh, Srijan Kumar

TL;DR

This paper investigates the role of the cross-modal projection layer in multimodal large language models (MLLMs) when fine-tuned for domain-specific tasks. Surprisingly, they find that while fine-tuning significantly improves MLLM performance on domain-specific image classification, the projection layer itself doesn't become better at extracting relevant visual attributes. Instead, the LLM component appears to be primarily responsible for learning these domain-specific features.

What Problem Does This Paper Address?

Modern multimodal large language models (MLLMs) like LLaVA and GPT-4(V) are capable of understanding and discussing images. However, their general-purpose training often limits their effectiveness in specialized domains like agriculture or dermatology. To address this, researchers fine-tune these models. A common MLLM architecture consists of an image encoder, a cross-modal projection network, and a large language model (LLM). It's widely assumed that the projection layer is crucial for translating visual information into a format the LLM can understand. This paper questions this assumption by asking: when we fine-tune an MLLM for a specific domain, what exactly is the projection layer learning, and what is the LLM learning? Understanding this is key to designing more efficient and interpretable MLLMs.

Methods

The researchers employed two primary fine-tuning strategies on the LLaVA-1.5 model:

  1. Fine-tuning the Projection Layer Only (FT-Proj): In this setting, the cross-modal projection layer (a small MLP) was updated, while the LLM parameters were kept frozen.
  2. End-to-End Fine-tuning (FT-E2E): Here, both the projection layer and the LLM parameters were updated concurrently.

To assess what the projection layer was learning, they developed a novel method: they trained an independent, simple MLP classifier using the post-projection representation (the output of the projection layer) from each fine-tuning setting. If the projection layer was effectively capturing domain-specific visual attributes, this independent MLP should perform well on domain-specific tasks when trained on these enhanced representations.

Datasets

The study utilized four domain-specific image classification datasets:

  • Agriculture: Plant-Doc dataset (17 classes of leaf diseases).
  • Textures: A dataset with 47 texture-related classes.
  • Dermatology: DermNet dataset (23 types of skin diseases).
  • Humanitarian: CrisisMMD dataset (4 categories related to crisis situations), considered closest to natural/internet images.

Key Insights and Ideas

  • Projection Layer Isn't the Primary Learner of Domain Attributes: The most striking finding is that fine-tuning MLLMs for domain-specific tasks doesn't necessarily improve the projection layer's ability to extract relevant visual features. In fact, the "richness" of the post-projection representation (as measured by an independent MLP classifier) often decreased after fine-tuning.
  • LLM is the Real Domain Expert: The results strongly suggest that the LLM component, whether frozen or fine-tuned, is where the domain-specific visual attributes are primarily modeled and understood. The projection layer seems to act more as a conduit, adapting to better utilize the LLM's existing capabilities rather than learning new visual concepts itself.
  • Reinterpreting the Projection's Role: This challenges the conventional view of the projection layer as the sole bridge for visual-to-textual alignment. It suggests its role might be more about facilitating the LLM's access to visual information rather than actively encoding domain-specific visual semantics.

Key Results

  • Performance Boost: Both FT-Proj and FT-E2E fine-tuning strategies significantly improved the MLLM's domain-specific image classification performance compared to zero-shot LLaVA-1.5. FT-E2E yielded the best performance, likely due to the larger number of trainable parameters (~7B vs. ~20M for FT-Proj).
  • Diminished Post-Projection Richness: In both fine-tuning settings (FT-Proj and FT-E2E), the independent MLP classifier trained on the post-projection representations performed worse than when trained on the original LLaVA-1.5's post-projection representations. This indicates a decrease in the "domain-specific richness" of these representations.
  • Image-Only Models Outperform FT-Proj: Interestingly, for three out of the four tasks (Agriculture, Textures, Dermatology), a simple MLP trained directly on CLIP image embeddings (the input to the projection layer) outperformed the MLLM fine-tuned only on the projection layer (FT-Proj).

Limitations and Future Work

  • Limited Architecture: The study focused on a specific MLLM architecture (LLaVA-1.5) and a particular type of projection layer (MLP). Future work could explore other projection schemes like trainable linear layers, gated cross-attention, or Q-Formers found in models like Flamingo and InstructBLIP.
  • Interpretability Methods: While the MLP classifier provides a good estimate of richness, future research could employ more advanced interpretability techniques like concept bottleneck models or analyze mutual information.
  • Holistic Evaluation: The authors suggest that future evaluations should consider both domain-specific capabilities and general-purpose performance.
  • Broader Social Impact: While this specific work has no foreseen negative social impacts, the authors acknowledge that LLMs can inherit and introduce biases, which needs to be considered in future domain-specific MLLM development.
  • Future Research Directions: The authors aim to interpret layer- and neuron-level contributions in LLMs for multimodal reasoning. They also emphasize the need to consider modality-specific reasoning alongside LLM biases in future work.

Summary updated

Abstract

Multimodal large language models (MLLMs) like LLaVA and GPT-4(V) enable general-purpose conversations about images with the language modality. As off-the-shelf MLLMs may have limited capabilities on images from domains like dermatology and agriculture, they must be fine-tuned to unlock domain-specific applications. The prevalent architecture of current open-source MLLMs comprises two major modules: an image-language (cross-modal) projection network and a large language model. It is desirable to understand the roles of these two modules in modeling domain-specific visual attributes to inform the design of future models and streamline the interpretability efforts on the current models. To this end, via experiments on 4 datasets and under 2 fine-tuning settings, we find that as the MLLM is fine-tuned, it indeed gains domain-specific visual capabilities, but the updates do not lead to the projection extracting relevant domain-specific visual attributes. Our results indicate that the domain-specific visual attributes are modeled by the LLM, even when only the projection is fine-tuned. Through this study, we offer a potential reinterpretation of the role of cross-modal projections in MLLM architectures. Project webpage: https://claws-lab.github.io/projection-in-MLLMs/
BibTeX
@inproceedings{verma2024cross,
  author = {Gaurav Verma and Minje Choi and Kartik Sharma and Jamelle Watson-Daniels and Sejoon Oh and Srijan Kumar},
  title = {{Cross-Modal Projection in Multimodal LLMs Doesn't Really Project Visual Attributes to Textual Space}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-short.60},
}
Figure