Learning Network Architecture for Open-Set Recognition

Xuelin Zhang, Xuelian Cheng, Donghao Zhang, C. Paul Bonnington, Zongyuan Ge

Abstract

Most deep learning models operate under a "closed-set" assumption, where testing classes are identical to training classes. However, real-world applications require Open-Set Recognition (OSR) systems capable of classifying Known Known Classes (KKCs) while identifying and rejecting Unknown Unknown Classes (UUCs). While prior research has focused on loss functions and calibration, the impact of network architecture on OSR remains under-explored. This paper presents the first attempt to use Neural Architecture Search (NAS) specifically for OSR. By combining a VAE-based contrastive learning framework with a novel "Pseudo Auxiliary Searching" (PAS) strategy, we discover efficient network architectures that significantly outperform human-designed models. Our searched models achieve state-of-the-art performance across five benchmarks while utilizing up to 78% fewer parameters than existing methods.

Introduction

In critical domains like autonomous driving or medical diagnosis, the ability to recognize the "unknown" is as vital as classifying the "known." Standard architectures like ResNet or VGG are optimized for closed-set discriminative power, which often leads to overconfident predictions on out-of-distribution samples—a phenomenon known as "open space risk."

We argue that the physical structure of a network—its connections and operations—inherently affects the smoothness and compactness of feature boundaries. To address this, we propose an end-to-end hierarchical NAS pipeline designed to search for architectures that naturally minimize open space risk by creating more robust boundaries between known and unknown data clusters.

Methodology

Hierarchical NAS for OSR

Our search space is hierarchical, encompassing both "cell-level" operations (e.g., specific convolution types) and "network-level" paths that control spatial resolution and downsampling. This allows the model to optimize for the discriminative depth required for OSR.

Pseudo Auxiliary Searching (PAS)

A core challenge in searching for OSR architectures is the absence of unknown classes during the search phase. We introduce the PAS strategy to simulate this environment. By splitting the training set and treating a portion of known classes as "pseudo" unknowns, the super-net is forced to explore architectures that are specifically effective at creating boundaries between knowns and simulated outliers.

VAE Contrastive Learning

To regularize the search space, we utilize a Variational Autoencoder (VAE) framework. This helps the model disentangle class attributes (discriminative features) from instance attributes (nuisance variables like lighting or pose). By fitting these features to a Compact Abating Probability (CAP) model, the network ensures that the probability of a class decreases as a sample moves further away from known data clusters in the latent space.

The proposed model pipeline integrates a multi-branch feature encoder with a VAE-based attribute disentanglement framework.

Experiments and Results

Experimental Setup

We evaluated the discovered architecture on five standard OSR benchmarks: MNIST, SVHN, CIFAR10, and two CIFAR-based extensions (C+10 and C+50) that use CIFAR100 classes as unknowns. The architecture search was conducted on CIFAR10, and the resulting configuration was transferred to the other datasets to test generalization.

Performance and Efficiency

Our NAS-generated model outperformed prior state-of-the-art (SOTA) models across nearly all benchmarks. Notably, on CIFAR10, it improved the AUROC by 12% compared to the previous best method, GCM-CF. Furthermore, the discovered architecture is highly efficient, achieving these results with 78% fewer parameters and 42.7% fewer FLOPs than GCM-CF.

Quantitative comparison of the proposed NAS method against existing OSR approaches across multiple datasets.

Robustness to Openness

In evaluations against varying degrees of "openness" (the ratio of unknown to known classes), our model demonstrated superior stability. While standard CNNs and OpenMax approaches show rapid performance degradation as more unknown classes are introduced, the NAS-optimized structure maintains a significantly higher F1-score, proving its robustness in increasingly complex open-set environments.

Discussion and Conclusion

This study demonstrates that architecture is a "first-class citizen" in Open-Set Recognition. The success of the PAS strategy suggests that simulating the unknown during the search phase is a powerful meta-learning tool for OSR. While we observed a slight performance dip on MNIST due to the domain gap between RGB search data and grayscale test data, the overall generalization across SVHN and CIFAR-based tasks is exceptional.

Future work will focus on scaling this approach to larger datasets like ImageNet and exploring the simultaneous search of both the encoder and decoder components to further refine the attribute disentanglement process.

Summary updated

Abstract

Given the incomplete knowledge of classes that exist in the world, Open-set Recognition (OSR) enables networks to identify and reject the unseen classes after training. This problem of breaking the common closed-set assumption is far from being solved. Recent studies focus on designing new losses, neural network encoding structures, and calibration methods to optimize a feature space for OSR relevant tasks. In this work, we make the first attempt to tackle OSR by searching the architecture of a Neural Network (NN) under the open-set assumption. In contrast to the prior arts, we develop a mechanism to both search the architecture of the network and train a network suitable for tackling OSR. Inspired by the compact abating probability (CAP) model, which is theoretically proven to reduce the open space risk, we regularize the searching space by VAE contrastive learning. To discover a more robust structure for OSR, we propose Pseudo Auxiliary Searching (PAS), in which we split a pretended set of know-unknown classes from the original training set in the searching phase, hence enabling the super-net to explore an effective architecture that can handle unseen classes in advance. We demonstrate the benefits of this learning pipeline on 5 OSR datasets, including MNIST, SVHN, CIFAR10, CIFARAdd10, and CIFARAdd50, where our approach outperforms prior state-of-the-art networks designed by humans. To spark research in this field, our code is available at https://github.com/zxl101/NAS OSR.
BibTeX
@inproceedings{zhang2022learning,
  author = {Xuelin Zhang and Xuelian Cheng and Donghao Zhang and C. Paul Bonnington and Zongyuan Ge},
  title = {{Learning Network Architecture for Open-Set Recognition}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i3.20246},
}
Figure