Saving for the future: Enhancing generalization via partial logic regularization

Zhaorui Tan, Yijie Hu, Xi Yang, Qiufeng Wang, Anh Nguyen, Kaizhu HuangarXiv 2025 · DOI · arXiv

Abstract

Standard deep learning models typically operate under a closed-world assumption, where all possible classes are known at training time. When deployed in real-world scenarios, these models often struggle with "unknown" objects, either misclassifying them as known categories or overwriting old knowledge to accommodate new ones. This paper introduces PL-Reg (Partial-Logic Regularization), a novel training technique that shifts from traditional binary logic (True/False) to partial logic (True/False/Undefined). By allowing models to "save" part of their feature space for future, undefined categories, PL-Reg significantly improves performance in open-world and incremental learning tasks.

The Problem: Semantic Overlap and Over-Explanation

In standard computer vision, models are trained on a fixed set of "known" classes. However, real-world deployment frequently involves "unknown" classes (e.g., a person appearing in a pet-classifier). Current logic-based regularization methods, such as the previous L-Reg, utilize "sentential logic," which forces the model to use its entire representational capacity to describe known classes.

This leads to two major hurdles:

  1. Semantic Overlap: The model "over-explains" known objects using features that should ideally belong to unknown ones. For example, if a model uses a "mouth" feature to define a dog, it may get confused when it sees a human, who also has a mouth.
  2. Catastrophic Forgetting: In incremental learning, models often overwrite existing feature mappings to make room for new classes because no "empty" space was reserved.
Conceptual comparison between Sentential Logic (top) and the proposed Partial Logic (bottom). Partial logic prevents the model from assigning undefined logic to known classes, preserving feature space for future unknown categories.

As shown in the figure above, partial logic allows the model to ignore non-essential overlapping features for known classes, keeping the representation "cleaner" and reserving unique "vocabulary" for when it encounters new classes later.

Methods: The PL-Reg Mechanism

The core innovation of PL-Reg is the transition from Sentential Logic (binary: 0 or 1) to Partial Logic (three-valued: 0, 1, or 2, where 2 represents "undefined"). This is implemented through a lightweight regularization framework that can be integrated into existing architectures.

Architecture and Strategy

  • Partial Logic Mask Generator: A lightweight linear layer generates a mask ($M$) that splits the model's internal feature embeddings into two distinct parts: those with a "defined" meaning for current known classes and those that remain "undefined."
  • Dual-Loss Strategy:
    • Classification Loss ($L_{P1}$): This forces the model to distinguish between defined and undefined logic formulas, effectively telling the model which features to use for current knowledge and which to keep empty.
  • Diversity Loss ($L_{P2}$): This prevents the mask from collapsing (e.g., making everything undefined) by ensuring the reserved space is diverse across different dimensions.
The PL-Reg architecture: A mask generator splits feature embeddings into defined (red) and undefined (blue) components, which are then processed by a partial logic classifier.

Experimental Results

The authors validated PL-Reg across three challenging paradigms: Generalized Category Discovery (GCD), Multi-Domain GCD (mDG+GCD), and Class Incremental Learning (CIL), using datasets like CIFAR-100, ImageNet, and DomainNet.

Key Findings

  • Superior Generalization: On GCD tasks, PL-Reg improved accuracy for unknown classes to 63.7%, outperforming both the baseline (59.9%) and the previous logic-based champion, L-Reg (62.7%).
  • Robustness to Domain Shift: In multi-domain scenarios where both object categories and artistic styles change, PL-Reg achieved a significant +7.87% improvement in unknown class accuracy over state-of-the-art methods.
  • Mitigating Forgetting: In incremental learning tests on CIFAR-100, PL-Reg helped models maintain higher accuracy across 11 sequential sessions by preserving "room" for new classes in every step.

Theoretical and Qualitative Insights

Beyond empirical results, the authors provide a formal proof (Proposition 1) showing that a model using partial logic inherently possesses a lower generalization loss when encountering unseen data compared to binary logic models.

Qualitatively, GradCAM visualizations revealed that while previous models focused heavily on overlapping features (like a dog's mouth), PL-Reg encourages the model to focus on more discriminative, class-specific features. This "cleaner" representation treats unused feature space as a "savings account" for the future rather than mere noise.

Limitations and Future Work

While highly effective, PL-Reg does introduce a slight trade-off; in some datasets, a minor dip in "known" class accuracy was observed as the model reserved space for the future. Additionally, the current mathematical formulation assumes feature independence. Future research will focus on extending partial logic to interdependent semantic embeddings, regression tasks, and Large Language Models (LLMs) to handle ambiguous or out-of-distribution prompts.

Summary updated

Abstract

Generalization remains a significant challenge in visual classification tasks, particularly in handling unknown classes in real-world applications. Existing research focuses on the class discovery paradigm, which tends to favor known classes, and the incremental learning paradigm, which suffers from catastrophic forgetting. Recent approaches such as the L-Reg technique employ logic-based regularization to enhance generalization but are bound by the necessity of fully defined logical formulas, limiting flexibility for unknown classes. This paper introduces PL-Reg, a novel partial-logic regularization term that allows models to reserve space for undefined logic formulas, improving adaptability to unknown classes. Specifically, we formally demonstrate that tasks involving unknown classes can be effectively explained using partial logic. We also prove that methods based on partial logic lead to improved generalization. We validate PL-Reg through extensive experiments on Generalized Category Discovery, Multi-Domain Generalized Category Discovery, and long-tailed Class Incremental Learning tasks, demonstrating consistent performance improvements. Our results highlight the effectiveness of partial logic in tackling challenges related to unknown classes.
BibTeX
@misc{tan2025saving,
  author = {Zhaorui Tan and Yijie Hu and Xi Yang and Qiufeng Wang and Anh Nguyen and Kaizhu Huang},
  title = {{Saving for the future: Enhancing generalization via partial logic regularization}},
  year = {2025},
  eprint = {2508.15317},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.15317},
}
Figure