Introduction
Zero-shot (0shot) text classification has emerged as a powerful paradigm, allowing models to categorize text into arbitrary classes without task-specific training data. However, the most accurate implementations—typically cross-encoders based on Natural Language Inference (NLI) or Next Sentence Prediction (NSP)—suffer from a significant computational bottleneck. These models require a separate forward pass for every potential label, leading to linear scaling of inference time with respect to the label space size. For datasets with dozens of categories, this overhead makes real-time deployment and large-scale processing prohibitively expensive and environmentally taxing.
Methodology: Conformalized Label Pruning
To address this inefficiency, we propose a two-stage framework that leverages Conformal Prediction (CP) to prune the label space before invoking the heavy 0shot model.
The Two-Stage Pipeline
- Base Classifier (The Filter): A computationally inexpensive model, such as a token-overlap heuristic or a distilled BERT, performs an initial pass to generate scores for all possible labels.
- Conformal Prediction (The Guarantee): Rather than selecting a fixed number of top-K labels, the framework applies CP to these scores. This statistical method produces a dynamic set of candidate labels. The size of this set varies based on the model's uncertainty, but it provides a rigorous mathematical guarantee: the true label (as determined by the expensive 0shot model) will be contained within the set with a probability of at least $1 - \alpha$, where $\alpha$ is a user-defined error tolerance.
- The 0shot Model (The Judge): The expensive NLI/NSP model is then executed only on the reduced set of candidate labels.
Self-Supervised Calibration
A notable feature of this approach is its zero-shot nature. The system does not require human-labeled data for calibration. Instead, the expensive 0shot model is used to label a small subset of unlabeled text. These "silver" labels are then used to calibrate the base classifier's scores, ensuring the conformal guarantees hold without any manual annotation effort.
Experimental Setup
The framework was evaluated across five diverse datasets: Intent classification (SNIPS, ATIS, HWU64) and Topic classification (AG’s News, Yahoo! Answers). The "heavy" 0shot models included bart-large-mnli for NLI-based tasks and bert-base-uncased for NSP-based tasks.
Results and Discussion
Statistical Reliability
The primary requirement for any pruning strategy is that it must not discard the correct label. The conformal framework ensures this through its calibration process. As shown in the empirical coverage analysis, the actual probability of the true label being in the prediction set closely matches the target nominal coverage.
Efficiency and Scaling
The efficiency gains are driven by the reduction in the number of labels the heavy model must process. On average, the CP framework reduced the label space by 41% to 43%. This reduction translated into average inference speedups of 25.6% for NLI models and 22.2% for NSP models.
The benefits are most pronounced in high-cardinality datasets. In the HWU64 dataset, which contains 64 labels, the overhead of the base classifier is negligible compared to the massive savings achieved by skipping dozens of expensive forward passes.
The Complexity Trade-off
The choice of base classifier involves a trade-off between speed and precision. Simple heuristics like token overlap are extremely fast but may produce larger candidate sets. Conversely, more sophisticated models like distilled BERT produce smaller, more precise sets but require more initial computation. For datasets with many labels, the investment in a slightly more complex base classifier often yields better overall throughput.
Conclusion
By integrating Conformal Prediction into the zero-shot classification pipeline, we can significantly reduce the computational cost of high-accuracy models without sacrificing reliability. The framework's ability to operate without human-labeled data makes it a practical solution for scaling zero-shot classification to complex, multi-label domains. Future work will explore extending this methodology to large generative models and addressing potential biases introduced during the filtering stage.
Summary updated