Zero-Shot Classification by Logical Reasoning on Natural Language Explanations

Chi Han, Hengzhi Pei, Xinya Du, Heng Ji

Abstract

Humans possess the remarkable ability to classify unseen categories by reasoning over natural language explanations. This capability stems from the compositional nature of language, allowing the combination of known attributes to describe novel concepts. Inspired by this, we propose CLORE (Classification by LOgical Reasoning on Explanations), a framework that tackles zero-shot classification (ZSC) by logically parsing and reasoning on natural language explanations. Unlike previous methods that treat text as implicit feature vectors, CLORE explicitly parses explanations into logical structures and reasons over them to produce classification scores. Our results demonstrate that CLORE outperforms baselines, particularly in tasks requiring complex logical reasoning, while providing interpretable rationales and showing resilience to linguistic biases.

1. Introduction

Zero-shot classification (ZSC) requires models to identify objects from categories not encountered during training. To achieve this, models utilize "side information," typically in the form of natural language descriptions. Traditional approaches often map these descriptions to a single embedding vector (e.g., using BERT or CLIP), which is then compared to the input data. However, this "black-box" approach fails to capture the compositional logic of language. Humans do not just compare similarity; they decompose descriptions into specific attributes and verify their presence through logical reasoning.

CLORE supports modality-flexible inputs, parsing structured, textual, or visual data into a logical reasoning process.

2. The CLORE Framework

CLORE is a two-stage neuro-symbolic framework designed to mimic human-like reasoning for classification.

2.1 Logical Parsing

The framework first identifies key attribute phrases within an explanation using an attention-based Gated Recurrent Unit (GRU). It then maps the sentence to a logical template (e.g., $Attribute_1 \land Attribute_2$). To ensure the model remains end-to-end differentiable, it predicts a probability distribution over a set of predefined logical tree templates rather than selecting a single hard parse.

2.2 Logical Reasoning

Matching scores are calculated between the detected attributes and the input data (such as image patches or table cells) using cosine similarity. These scores are then combined using probabilistic "fuzzy logic" operators: the min operator for AND junctions and the max operator for OR junctions.

The logical reasoning stage combines attribute matching scores ($s_n$) using fuzzy logic operators to produce a final classification score.

3. Experimental Results

We evaluated CLORE on the CLUES benchmark (tabular data) and a newly introduced CUB-Explanations benchmark (visual data with Wikipedia descriptions).

3.1 Performance and Interpretability

CLORE achieved 57.4% accuracy on the CLUES benchmark, outperforming the ExEnt baseline (54.8%). In visual ZSC on the CUB dataset, CLORE reached 39.1% accuracy, significantly higher than a fine-tuned CLIP baseline (29.9%). The performance gap widened as the logical complexity of the explanations increased, suggesting that explicit reasoning is crucial for handling sophisticated descriptions.

Beyond accuracy, CLORE provides built-in interpretability. By visualizing the attention maps and the logical tree execution, users can see exactly which parts of the input data satisfied specific attributes in the explanation. Furthermore, CLORE proved robust against "linguistic noise," maintaining stable performance even when explanations were modified with doubting or verbose phrases.

4. Conclusion

CLORE demonstrates the power of integrating logical reasoning into zero-shot classification. By treating language as a structured program rather than a simple vector, the model achieves better performance, higher interpretability, and greater robustness. Future work will focus on expanding the logical scope to include relational and temporal dependencies.

Summary updated

Abstract

Humans can classify data of an unseen category by reasoning on its language explanations. This ability is owing to the compositional nature of language: we can combine previously seen attributes to describe the new category. For example, we might describe a sage thrasher as "it has a slim straight relatively short bill, yellow eyes and a long tail", so that others can use their knowledge of attributes "slim straight relatively short bill", "yellow eyes" and "long tail" to recognize a sage thrasher. Inspired by this observation, in this work we tackle zero-shot classification task by logically parsing and reasoning on natural language expla-nations. To this end, we propose the framework CLORE (Classification by LOgical Reasoning on Explanations). While previous methods usually regard textual information as implicit features, CLORE parses explanations into logical structures and then explicitly reasons along thess structures on the input to produce a classification score. Experimental results on explanation-based zero-shot classification benchmarks demonstrate that CLORE is superior to baselines, which we further show mainly comes from higher scores on tasks requiring more logical reasoning. We also demonstrate that our framework can be extended to zero-shot classification on visual modality. Alongside classification decisions, CLORE can provide the logical parsing and reasoning process as a clear form of rationale. Through empirical analysis we demonstrate that CLORE is also less affected by linguistic biases than baselines.
BibTeX
@inproceedings{han2023zero,
  author = {Chi Han and Hengzhi Pei and Xinya Du and Heng Ji},
  title = {{Zero-Shot Classification by Logical Reasoning on Natural Language Explanations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  url = {https://openreview.net/forum?id=iP5DxrB7tC},
}
Figure