TL;DR
Despite their impressive capabilities on complex multimodal reasoning tasks, popular Large Language and Vision Models (LLVMs) exhibit striking perceptual anomalies. This paper shows that LLVMs process images globally (showing permutation invariance to patch order), solve visual math questions via commonsense language shortcuts without actually reading the charts, and suffer severe catastrophic forgetting of basic visual recognition skills after multimodal alignment.
What Problem Does This Paper Address?
Most modern LLVMs (such as the LLaVA series) rely on "model stitching": combining a pre-trained Vision Transformer (ViT) and a Large Language Model (LLM) via a simple cross-modal projection layer. While these models excel at complex reasoning benchmarks, they notoriously fail at basic visual perception tasks that are trivial for humans.
This paper investigates how LLVMs actually perceive images: Do they preserve the properties of their underlying vision encoders? How do they handle spatial disruption and occlusion? Are they genuinely grounding their reasoning in visual details, or are they relying on textual and commonsense shortcuts?
Methods
The authors systematically probe LLVM representations and behaviors using several perturbation techniques:
- Patch Permutation & Spatial Jigsaw Shuffling: Testing permutation invariance by randomly shuffling visual patch token sequences before feeding them into the LLM, as well as applying spatial jigsaw shuffling at varying grid sizes ($2\times2$, $4\times4$, $8\times8$, $14\times14$).
- Synthetic Regeneration (Information Distortion): Generating descriptions of benchmark images using an LLVM and re-rendering them with SDXL-Lightning to strip exact numerical, chart, or formula layouts while retaining high-level semantics.
- Occlusion Analysis (PatchDrop): Masking out patches via Random PatchDrop, Salient PatchDrop, and Non-Salient PatchDrop (guided by DINO attention maps).
- Mutual Nearest-Neighbor (MNN) Alignment: Evaluating whether cross-modal alignment between LLMs and vision representations is preserved across instruction tuning, under the Platonic Representation Hypothesis.
- Inference-Time "Sharpness" Importance Score: Measuring layer-wise and modality-wise importance by adding bounded uniform noise to specific token positions or layer inputs and measuring downstream performance drops.
Datasets
The authors evaluate across diverse vision-language benchmarks:
- Multimodal Evaluation Benchmarks: MMVP, Q-Bench, MME, MMStar, MM-Vet, LLaVA-Bench (LLaVA-W), MathVista, ScienceQA-Image (SQA-IMG), ChartQA, AI2D, SEED-Bench, GSM8K (text-only 8-shot CoT).
- Zero-Shot Classification (Alignment/Forgetting): Caltech101, CIFAR-100, Food101, Oxford-IIIT Pets, Country211, EuroSAT, FGVC-Aircraft.
- Image-Text / Representation Alignment: DOCCI (long descriptive captions of natural scenes).
- Captioning Benchmarks: COCO-Captions, NoCaps, TextCaps.
Key Insights and Ideas
- LLVMs are surprisingly permutation-invariant: Shuffling the sequence of visual patch tokens causes almost no performance drop on perceptual benchmarks (<1% average drop for LLaVA-1.5 across 10 benchmarks). Because backpropagation loss is derived purely from autoregressive text targets, the model processes visual tokens as a global "bag of tokens."
- Visual math can often be solved "blind": On synthetic MathVista images where charts and formulas were scrambled or removed, models still achieved strong scores. A major reason: many questions ask "What is the smallest value?", and models defaulted to answering
1via language prior biases rather than reading the plot. - Catastrophic forgetting of basic vision: Stitching a vision encoder to an LLM severely degrades raw image classification abilities. Both LLaVA-1.5 and LLaVA-NeXT drop by nearly 30% on zero-shot image classification compared to their standalone CLIP vision encoders. Cross-modal representation alignment to natural scenes (DOCCI) also drops after visual instruction tuning.
- Lower layers do the visual work: Layer-importance scoring shows that the bottom 20–25% of LLM transformer blocks (< layer 8) are dominated by visual processing and have the largest impact on downstream multimodal accuracy. Higher layers gradually shift focus almost entirely to text generation.
- Strong center bias: Patch importance analysis reveals that LLVMs place massive weight on central image tokens, while edge patches contribute negligibly to final performance.
Key Results
- Permutation Invariance (Table 1 & 4):
- LLaVA-1.5: Average performance across 10 benchmarks dropped by only 0.59% under random patch token shuffling (even slightly improving on MMVP: +1.33%).
- LLaVA-NeXT: Average drop of 2.71%.
- LLaVA-OneVision: Showed a larger average drop of 9.40%, driven by severe degradation on text/geometry-rich tasks like ChartQA (▼35.26%) and MathVista (▼8.30%).
- Visual Math Shortcuts (Table 2 & 6):
- On synthesized/degraded MathVista images, LLaVA-1.5's accuracy dropped by merely 1.8% (from 24.7% to 22.9%).
- When given free-form questions on degraded images, LLaVA-1.5 outputted the number
"1"81.0% of the time. - Degradation of Perceptual Representations (Table 3):
- Zero-shot classification average accuracy dropped from 67.27% (standalone CLIP-ViT-L/336px) to 37.64% (LLaVA-1.5), a relative drop of nearly 30% (e.g., EuroSAT dropped from 58.90% to 11.72%).
- Occlusion Robustness (Figure 6 & 12):
- LLVMs inherit the strong occlusion robustness of ViTs; on AI2D, models retained high task accuracy even with 50%–70% of patches masked out.
Limitations and Future Work
- Model Coverage: The primary focus was on the LLaVA family (with preliminary extensions to Qwen2-VL and Fuyu); broader architectures remain to be analyzed.
- Evaluation Bias: Many benchmarks rely on multiple-choice formats where language shortcuts and selection bias obscure genuine perceptual failures.
- Future Directions:
- Build multi-turn, interactive, and strictly vision-grounded evaluation benchmarks with free-form answers to penalize blind guessing.
- Mitigate catastrophic forgetting via balanced rehearsal or localized-aware projection modules (e.g., convolution-based projectors like HoneyBee).
- Train lower LLM layers with targeted visual objectives (e.g., masked autoencoding or layer-traversal architectures like TroL) to preserve fine-grained cross-modal grounding.
Summary updated