Abstract
Humans can classify data of an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen attributes to describe the new category. For example, we might describe a sage thrasher as “it has a slim straight relatively short bill, yellow eyes and a long tail”, so that others can use their knowledge of attributes “slim straight relatively short bill”, “yellow eyes” and “long tail” to recognize a sage thrasher. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language explanations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses explanations into logical structures and then explicitly reasons along this structure on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, which we show is mainly due to higher scores on tasks requiring more logical reasoning. We also demonstrate that our framework can be extended to zero-shot classification on visual modality. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a clear form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.
Introduction
Zero-shot classification (ZSC) is the challenging task of recognizing categories that a model has never encountered during training. Most current state-of-the-art approaches, such as CLIP-based encoders, treat category descriptions as "implicit features," converting entire sentences into single mathematical vectors. However, this approach often fails to capture the compositional nature of language. If a bird is described as having a "slim bill AND yellow eyes," a human logically checks for both attributes.
To address this, we introduce CLORE (Classification by LOgical REasoning on Explanations), a framework that shifts ZSC from "black-box" feature matching to explicit symbolic reasoning. By parsing natural language into logical structures, CLORE becomes more robust to linguistic noise and more effective at handling complex descriptions.
The CLORE Framework
CLORE operates in two main stages: Logical Parsing and Logical Reasoning.
Logical Parsing
The model uses an attention-based Gated Recurrent Unit (GRU) to identify "attribute candidates" within an explanation (e.g., "yellow eyes"). It then maps the explanation to a logical template, such as a binary tree of AND/OR operators. For example, the description of a sage thrasher is parsed into a logical formula: Slim_straight_short_bill(X) ∧ Yellow_eyes(X) ∧ Long_tail(X).
Logical Reasoning
Once parsed, the model performs reasoning in three steps:
- Attribute Matching: It calculates similarity scores between the input (such as image patches or table rows) and the extracted attributes.
- Probabilistic Logic: The model traverses the logical tree bottom-up using "fuzzy logic" operations—specifically
min(s1, s2)for AND andmax(s1, s2)for OR—to aggregate attribute scores into a final classification. - Certainty Weighting: An MLP predicts a "certainty value" for each explanation, allowing the model to prioritize definitive descriptions over vague ones (e.g., those using "maybe").
Experimental Evaluation
We evaluated CLORE on two distinct benchmarks:
- CLUES: A benchmark for zero-shot classification on structured (tabular) data.
- CUB-Explanations: A new benchmark introduced in this work, which augments the CUB-200-2011 bird dataset with Wikipedia-sourced natural language explanations focusing on visual attributes like plumage and beak shape.
Key Results
- Superior Accuracy: On the CLUES-Real dataset, CLORE achieved 57.4% accuracy, outperforming the best baseline, ExEnt (54.8%). In the visual domain, CLORE achieved a harmonic mean ($ACC_H$) of 49.1%, compared to 41.3% for a fine-tuned CLIP model.
- Robustness to Bias: CLORE is significantly less affected by linguistic biases. While standard transformers see performance drops when faced with added punctuation or filler text, CLORE remains stable because its logical parser filters out non-essential noise.
- Interpretability: Unlike standard encoders, CLORE provides a clear "execution evidence" map, showing exactly which attributes matched the input and how they contributed to the final decision.
Conclusion
By forcing models to follow explicit logical structures, CLORE avoids over-relying on spurious correlations and linguistic biases. As explanations become more complex and compositional, CLORE’s performance gap over traditional baselines increases, demonstrating the power of symbolic reasoning in zero-shot learning.
Summary updated