Abstract
Humans possess the remarkable ability to classify unseen categories by reasoning over natural language explanations. This capability stems from the compositional nature of language, allowing the combination of known attributes to describe novel concepts. Inspired by this, we propose CLORE (Classification by LOgical Reasoning on Explanations), a framework that tackles zero-shot classification (ZSC) by logically parsing and reasoning on natural language explanations. Unlike previous methods that treat text as implicit feature vectors, CLORE explicitly parses explanations into logical structures and reasons over them to produce classification scores. Our results demonstrate that CLORE outperforms baselines, particularly in tasks requiring complex logical reasoning, while providing interpretable rationales and showing resilience to linguistic biases.
1. Introduction
Zero-shot classification (ZSC) requires models to identify objects from categories not encountered during training. To achieve this, models utilize "side information," typically in the form of natural language descriptions. Traditional approaches often map these descriptions to a single embedding vector (e.g., using BERT or CLIP), which is then compared to the input data. However, this "black-box" approach fails to capture the compositional logic of language. Humans do not just compare similarity; they decompose descriptions into specific attributes and verify their presence through logical reasoning.
2. The CLORE Framework
CLORE is a two-stage neuro-symbolic framework designed to mimic human-like reasoning for classification.
2.1 Logical Parsing
The framework first identifies key attribute phrases within an explanation using an attention-based Gated Recurrent Unit (GRU). It then maps the sentence to a logical template (e.g., $Attribute_1 \land Attribute_2$). To ensure the model remains end-to-end differentiable, it predicts a probability distribution over a set of predefined logical tree templates rather than selecting a single hard parse.
2.2 Logical Reasoning
Matching scores are calculated between the detected attributes and the input data (such as image patches or table cells) using cosine similarity. These scores are then combined using probabilistic "fuzzy logic" operators: the min operator for AND junctions and the max operator for OR junctions.
3. Experimental Results
We evaluated CLORE on the CLUES benchmark (tabular data) and a newly introduced CUB-Explanations benchmark (visual data with Wikipedia descriptions).
3.1 Performance and Interpretability
CLORE achieved 57.4% accuracy on the CLUES benchmark, outperforming the ExEnt baseline (54.8%). In visual ZSC on the CUB dataset, CLORE reached 39.1% accuracy, significantly higher than a fine-tuned CLIP baseline (29.9%). The performance gap widened as the logical complexity of the explanations increased, suggesting that explicit reasoning is crucial for handling sophisticated descriptions.
Beyond accuracy, CLORE provides built-in interpretability. By visualizing the attention maps and the logical tree execution, users can see exactly which parts of the input data satisfied specific attributes in the explanation. Furthermore, CLORE proved robust against "linguistic noise," maintaining stable performance even when explanations were modified with doubting or verbose phrases.
4. Conclusion
CLORE demonstrates the power of integrating logical reasoning into zero-shot classification. By treating language as a structured program rather than a simple vector, the model achieves better performance, higher interpretability, and greater robustness. Future work will focus on expanding the logical scope to include relational and temporal dependencies.
Summary updated