Probing Conceptual Understanding of Large Visual-Language Models

Madeline Chantry Schiappa, Michael Cogswell, Ajay Divakaran, Yogesh Singh Rawat

TL;DR

This paper introduces a new benchmarking suite designed to test whether large visual-language (V+L) models like CLIP truly "understand" the world or are simply high-dimensional pattern matchers. By probing models on relational, compositional, and contextual reasoning, the authors find that most state-of-the-art models lack a deep conceptual grasp, though a targeted fine-tuning approach called "patching" shows promise in bridging this gap.

What Problem Does This Paper Address?

Large V+L models have shown incredible performance on standard tasks like image retrieval and classification. However, the community has begun to suspect that these models rely on shortcuts—memorizing textures or co-occurrence statistics—rather than building internal "conceptual maps" similar to human understanding.

For instance, a model might correctly identify a "dog on a beach" because it has seen thousands of similar images, but it might fail to notice the absurdity if the dog were replaced by a toaster in the same setting. This paper seeks to quantify this "conceptual gap" by creating tasks that explicitly test if a model understands how objects relate to one another, how their attributes (like color or material) compose their identity, and how their surrounding context influences their presence.

Methods

The authors ground their evaluation in the cognitive science paradigm of "violation-of-expectation" (VoE). If a model has a conceptual understanding, its confidence should drop significantly when presented with an implausible scenario (e.g., a "parking lot on a car" instead of a "car in a parking lot").

Figure 1. Conceptual understanding of an existing V+L model. Here, CLIP failure to understand relational, compositional and con- textual reasoning is shown. This benchmark presents three datasets to evaluate V+L models on relational, compositional, and contextual understanding. They utilize image-text matching tasks with predicate, object/subject, compositions, or background swaps.

To address the weaknesses found, they propose a fine-tuning method for dual-stream models (like CLIP). They use selective negative sampling, where the model is trained to distinguish a correct image-text pair from "hard negatives" where only the relation or attribute is swapped. To prevent the model from losing its general knowledge (a problem known as "catastrophic forgetting"), they use a weight patching technique, linearly interpolating the weights of the fine-tuned model with the original pre-trained weights.

Datasets

The authors introduced three new probing datasets and one fine-tuning dataset:

  1. Probe-R (Relational): Derived from Visual Genome, it swaps predicates (e.g., "next to" vs. "on") or subjects to see if the model understands spatial and functional relationships.
  2. Probe-C (Compositional): Based on MS COCO, it swaps attributes with antonyms (e.g., "small dog" vs. "big dog") or swaps objects while keeping attributes the same.
  3. Probe-B (Contextual): Uses MS COCO to test reliance on background. It involves removing backgrounds or replacing them with random "fillers" (noise, gray, or random scenes) to see if object recognition fails without its usual context.
  4. RelComp: A new dataset created for the fine-tuning experiments, ensuring no overlap with the probing benchmarks.
Overview of the proposed benchmarks: Probe-R, Probe-C, and Probe-B.

Key Insights and Ideas

  • The Cross-Attention Advantage: Models that use "co-attention" or "cross-attention" (where the model looks at image and text features simultaneously, like BridgeTower or ViLT) perform significantly better on relational tasks than dual-stream models like CLIP. This suggests that inter-modality interaction is crucial for understanding relationships.
  • Texture vs. Shape Bias: The study confirms that CNN-based backbones are more sensitive to texture and patterns, while Transformer-based backbones (ViTs) are better at recognizing color and shape.
  • Context as a Distractor: Surprisingly, many models actually improve their recognition accuracy when the background is removed. This suggests that for these models, background context often acts as noise or a distractor rather than a helpful cue.
  • The "Whole" vs. "Parts": Models are much better at recognizing when an object is swapped than when a predicate (relation) is swapped. This indicates they often treat prompts as a "bag of words" rather than a coherent relational structure.

Key Results

The evaluation of ten major models (including CLIP, BLIP2, and FLAVA) revealed a widespread lack of conceptual understanding.

  • Relational Failure: When predicates were swapped to something unlikely, model confidence remained high for the wrong answer, indicating a lack of a "conceptual map" for what relations are physically possible.
  • Compositional Struggle: Most models performed twice as well when objects were switched compared to when attributes (compositions) were switched.
  • Fine-tuning Success: The authors' "CLIP Patched" model showed significant gains. For example, on Probe-R, accuracy jumped from 53.52% (baseline CLIP) to 71.20% (Patched VT). However, this came at the cost of a slight drop in general ImageNet accuracy (from 63.6% to 54.6%), highlighting a clear trade-off between specialized conceptual robustness and general performance.
Radar plot showing accuracy and mean confidence of different models on relational understanding.

Limitations and Future Work

The primary limitation acknowledged is the scale of the fine-tuning data. While RelComp improved robustness, it is tiny compared to the massive datasets used for initial pre-training. The authors also noted the "catastrophic forgetting" drift in ImageNet scores, suggesting that finding the perfect balance between conceptual understanding and general zero-shot performance remains an open challenge.

Future work could involve scaling up the RelComp approach or developing new architectures that inherently prioritize relational and compositional structures without the heavy computational cost of full cross-attention.

Summary updated

Abstract

In recent years large visual-language (V+L) models have achieved great success in various downstream tasks. However, it is not well studied whether these models have a conceptual grasp of the visual content. In this work we focus on conceptual understanding of these large V+L models. To facilitate this study, we propose novel benchmarking datasets for probing three different aspects of content understanding, 1) \textit{relations}, 2) \textit{composition}, and 3) \textit{context}. Our probes are grounded in cognitive science and help determine if a V+L model can, for example, determine if snow garnished with a man is implausible, or if it can identify beach furniture by knowing it is located on a beach. We experimented with many recent state-of-the-art V+L models and observe that these models mostly \textit{fail to demonstrate} a conceptual understanding. This study reveals several interesting insights such as that \textit{cross-attention} helps learning conceptual understanding, and that CNNs are better with \textit{texture and patterns}, while Transformers are better at \textit{color and shape}. We further utilize some of these insights and investigate a \textit{simple finetuning technique} that rewards the three conceptual understanding measures with promising initial results. The proposed benchmarks will drive the community to delve deeper into conceptual understanding and foster advancements in the capabilities of large V+L models. The code and dataset is available at: \url{https://tinyurl.com/vlm-robustness}
BibTeX
@inproceedings{schiappa2023probing,
  author = {Madeline Chantry Schiappa and Michael Cogswell and Ajay Divakaran and Yogesh Singh Rawat},
  title = {{Probing Conceptual Understanding of Large Visual-Language Models}},
  booktitle = {Workshops @ CVPR},
  year = {2023},
  url = {https://openreview.net/forum?id=MGdNsu5xCf},
}
Figure