Abstract
Standard deep learning models typically operate under a closed-world assumption, where all possible classes are known at training time. When deployed in real-world scenarios, these models often struggle with "unknown" objects, either misclassifying them as known categories or overwriting old knowledge to accommodate new ones. This paper introduces PL-Reg (Partial-Logic Regularization), a novel training technique that shifts from traditional binary logic (True/False) to partial logic (True/False/Undefined). By allowing models to "save" part of their feature space for future, undefined categories, PL-Reg significantly improves performance in open-world and incremental learning tasks.
The Problem: Semantic Overlap and Over-Explanation
In standard computer vision, models are trained on a fixed set of "known" classes. However, real-world deployment frequently involves "unknown" classes (e.g., a person appearing in a pet-classifier). Current logic-based regularization methods, such as the previous L-Reg, utilize "sentential logic," which forces the model to use its entire representational capacity to describe known classes.
This leads to two major hurdles:
- Semantic Overlap: The model "over-explains" known objects using features that should ideally belong to unknown ones. For example, if a model uses a "mouth" feature to define a dog, it may get confused when it sees a human, who also has a mouth.
- Catastrophic Forgetting: In incremental learning, models often overwrite existing feature mappings to make room for new classes because no "empty" space was reserved.
As shown in the figure above, partial logic allows the model to ignore non-essential overlapping features for known classes, keeping the representation "cleaner" and reserving unique "vocabulary" for when it encounters new classes later.
Methods: The PL-Reg Mechanism
The core innovation of PL-Reg is the transition from Sentential Logic (binary: 0 or 1) to Partial Logic (three-valued: 0, 1, or 2, where 2 represents "undefined"). This is implemented through a lightweight regularization framework that can be integrated into existing architectures.
Architecture and Strategy
- Partial Logic Mask Generator: A lightweight linear layer generates a mask ($M$) that splits the model's internal feature embeddings into two distinct parts: those with a "defined" meaning for current known classes and those that remain "undefined."
- Dual-Loss Strategy:
- Classification Loss ($L_{P1}$): This forces the model to distinguish between defined and undefined logic formulas, effectively telling the model which features to use for current knowledge and which to keep empty.
- Diversity Loss ($L_{P2}$): This prevents the mask from collapsing (e.g., making everything undefined) by ensuring the reserved space is diverse across different dimensions.
Experimental Results
The authors validated PL-Reg across three challenging paradigms: Generalized Category Discovery (GCD), Multi-Domain GCD (mDG+GCD), and Class Incremental Learning (CIL), using datasets like CIFAR-100, ImageNet, and DomainNet.
Key Findings
- Superior Generalization: On GCD tasks, PL-Reg improved accuracy for unknown classes to 63.7%, outperforming both the baseline (59.9%) and the previous logic-based champion, L-Reg (62.7%).
- Robustness to Domain Shift: In multi-domain scenarios where both object categories and artistic styles change, PL-Reg achieved a significant +7.87% improvement in unknown class accuracy over state-of-the-art methods.
- Mitigating Forgetting: In incremental learning tests on CIFAR-100, PL-Reg helped models maintain higher accuracy across 11 sequential sessions by preserving "room" for new classes in every step.
Theoretical and Qualitative Insights
Beyond empirical results, the authors provide a formal proof (Proposition 1) showing that a model using partial logic inherently possesses a lower generalization loss when encountering unseen data compared to binary logic models.
Qualitatively, GradCAM visualizations revealed that while previous models focused heavily on overlapping features (like a dog's mouth), PL-Reg encourages the model to focus on more discriminative, class-specific features. This "cleaner" representation treats unused feature space as a "savings account" for the future rather than mere noise.
Limitations and Future Work
While highly effective, PL-Reg does introduce a slight trade-off; in some datasets, a minor dip in "known" class accuracy was observed as the model reserved space for the future. Additionally, the current mathematical formulation assumes feature independence. Future research will focus on extending partial logic to interdependent semantic embeddings, regression tasks, and Large Language Models (LLMs) to handle ambiguous or out-of-distribution prompts.
Summary updated