Unveiling Multimodal Processing: Exploring Activation Patterns in Multimodal LLMs for Interpretability and Efficiency

Chuan Wu, Meng Su, Youxuan Fang, Shaolin ZhuEMNLP 2025

TL;DR

This paper introduces a novel method to transform dense multimodal large language models (MLLMs) into interpretable Mixture-of-Experts (MoE) architectures. By visualizing expert activation patterns across different modalities, they uncover specialized and shared processing units, demonstrating how this knowledge can be used to prune models for significant efficiency gains with minimal performance loss.

What Problem Does This Paper Address?

Recent advancements in Multimodal Large Language Models (MLLMs) have led to impressive capabilities in understanding and generating content across text, images, and audio. However, the internal workings of these complex models remain largely opaque, making it difficult to understand how they integrate information from diverse sources. This "black box" nature hinders debugging, robustness improvements, and efficient deployment, especially given the computational intensity of these models. The paper aims to demystify this by exploring the neural activation patterns within MLLMs when processing different modalities.

Methods

The core of the paper's methodology is inspired by recent work on multilingual LLMs, adapting it for the multimodal domain. They transform dense MLLMs into fine-grained Mixture-of-Experts (MoE) architectures. This involves:

  • Expert Construction: Parameters of the feed-forward network (FFN) layers are clustered into "experts" using balanced K-Means clustering. This is done on the up-projection layer, with corresponding neurons in the down-projection layer grouped accordingly. This process is "modality-aware" in that the resulting experts are later analyzed for their activation patterns across different modalities.
  • Cross-Layer Expert Selection: To account for varying activation magnitudes across layers, a Z-score normalization is applied to expert activation scores within each FFN layer. Experts are then ranked globally across all layers for a given input token, and the top-K% are considered "activated experts."
  • Quantification and Visualization: Activation frequencies for each expert are calculated for different modalities (text, visual, audio, and combinations). These frequencies are visualized as heatmaps, providing a global view of expert utilization across layers and modalities.

Datasets

The paper uses the following datasets for experiments:

  • Text: MMLU (Massive Multitask Language Understanding) benchmark.
  • Visual: LLaVA-OneVision-Data corpus.
  • Audio: LibriSpeech dataset.

Key Insights and Ideas

  • Modality-Specific vs. Shared Experts: MLLMs exhibit distinct patterns of expert activation. Some experts are highly activated by specific modalities (e.g., visual experts in shallower layers), while others are consistently activated across modalities, suggesting they handle core reasoning or abstract concepts.
  • Hierarchical Processing: Shallower layers tend to show more modality-specific activation, particularly for visual inputs, indicating early-stage feature extraction. Deeper layers show more convergence and shared expert activation, suggesting abstract reasoning and response generation.
  • "White Stripes" in Qwen2.5-Omni: The all-modal Qwen2.5-Omni model shows distinct "white stripes" (regions of low activation) in its shallow layers, with these stripes varying based on the input modality. This suggests highly specialized, modality-specific pathways in the earliest FFN layers.
  • High-Frequency Shared Experts (HF-SEs): These are few in number and concentrated in deeper layers, forming the core modality-agnostic reasoning backbone.
  • Low-Frequency Shared Experts (LF-SEs): These are more numerous and found in middle layers, indicating a flexible mechanism for handling specific cross-modal abstractions.
  • Pruning Efficacy: Identifying expert functions (shared vs. modality-specific) and their activation frequencies allows for targeted model pruning. This approach significantly outperforms random pruning and can achieve comparable or even superior performance to the original unpruned models.

Key Results

  • LLaVA-NeXT: Shows pronounced differences in activation between visual and textual modalities in shallower layers, with visual inputs receiving more activation. Deeper layers show more similarity, indicating abstract reasoning.
  • Qwen2.5-Omni: Exhibits modality-specific pathways in shallow layers, with distinct "white stripes" for different modalities. Activation patterns become more uniform in deeper layers.
  • Pruning: Function-based pruning (e.g., "Reasoning+text+visual" experts) on Qwen2.5-Omni achieved performance close to or exceeding the original model while reducing FFN parameter usage by 10-20% (~9-17% FLOPs reduction).
  • Frequency-based Pruning: Global pruning (unequal per-layer pruning) was found to be superior to equal per-layer pruning, highlighting the layer-wise differences in expert activation sparsity.

Limitations and Future Work

  • Limitations of Experimental Data: The study used two representative MLLMs and may not capture the full diversity of multimodal tasks (e.g., medical image-text integration, multilingual speech-text interaction). The datasets used might also limit the discovery of internal mechanisms.
  • Limitations in Cross-Modal Reasoning: The study primarily focused on activation patterns and their impact on performance. Deeper exploration of how models perform complex cross-modal reasoning is limited.
  • Future Work: The authors plan to extend their experiments to a broader range of models and datasets, explore more complex multimodal interaction settings, and investigate cross-modal reasoning mechanisms that better align with human cognitive processes.

Summary updated

BibTeX
@inproceedings{wu2025unveiling,
  author = {Chuan Wu and Meng Su and Youxuan Fang and Shaolin Zhu},
  title = {{Unveiling Multimodal Processing: Exploring Activation Patterns in Multimodal LLMs for Interpretability and Efficiency}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure