Abstract: We propose introspective convolutional networks (ICN) that emphasize the importance of having convolutional neural networks empowered with generative capabilities. We employ a reclassification-by-synthesis algorithm to perform training using a formulation stemmed from the Bayes theory. Our ICN tries to iteratively: (1) synthesize pseudo-negative samples; and (2) enhance itself by improving the classification. The single CNN classifier learned is at the same time generative --- being able to directly synthesize new samples within its own discriminative model. We conduct experiments on benchmark datasets including MNIST, CIFAR-10, and SVHN using state-of-the-art CNN architectures, and observe improved classification results.
Introduction
Deep learning classifiers, while powerful, often lack robustness and require vast amounts of labeled data. They can be easily fooled by adversarial examples—small perturbations that lead to confident but incorrect classifications. Traditional approaches to improve robustness involve data augmentation or training on "hard examples." However, these typically rely on external processes or separate generative models like Generative Adversarial Networks (GANs).
We introduce Introspective Convolutional Networks (ICNs), which empower a single discriminative CNN with generative capabilities. By allowing a classifier to "introspect" and synthesize its own hard negative examples, we can tighten decision boundaries and improve both accuracy and robustness without additional model parameters or separate generator networks.
Methodology: Reclassification-by-Synthesis
The core of the ICN framework is an iterative process called reclassification-by-synthesis. Unlike GANs, which utilize a two-network minimax game, ICN uses a single CNN that alternates between two primary steps:
- Reclassification Step: The CNN is trained using standard supervised learning on the current dataset, which includes both original training data and self-generated "pseudo-negatives."
- Synthesis Step: The network generates new pseudo-negatives. Starting from random noise, the algorithm uses Stochastic Gradient Descent (SGD) via backpropagation directly on the input image space to find samples that the current model classifies as positive with high confidence.
These synthesized samples are then labeled as negatives for the next reclassification round. This process forces the model to learn a more precise boundary by explicitly rejecting samples that "look" like the target class but are actually model-generated noise. This modernizes older Generative via Discriminative Learning (GDL) methods by using deep CNNs and efficient SGD backpropagation for sampling.
Experiments and Results
We evaluated ICN on standard image classification benchmarks: MNIST, CIFAR-10, and SVHN.
Synthesis Quality
As training progresses, the synthesized pseudo-negatives evolve from random noise into recognizable structures that increasingly resemble the target classes. This visualization confirms that the internal representations learned by a standard discriminative CNN are rich enough to generate data.
Classification Performance
ICN consistently outperforms baseline CNNs across all datasets.
- On MNIST, the error rate dropped from 0.77% (baseline) to 0.72% (ICN softmax).
- On CIFAR-10, using a ResNet-32 baseline, error dropped from 6.70% to 6.52%.
- On SVHN, error dropped from 2.01% to 1.95%.
Notably, ICN is particularly effective in low-data regimes, as the network effectively "hallucinates" its own supplementary training data to improve data efficiency.
Adversarial Robustness
A key benefit of the introspective approach is natural adversarial defense. By training on images designed to fool its current state, the model becomes more robust to external attacks. In tests using the Fast Gradient Sign Method (FGSM), ICN reduced adversarial errors by 38.7% compared to the baseline, achieved entirely through internal introspection.
Discussion and Future Work
ICN demonstrates that a classifier can be its own generator. By focusing on "hard negatives" rather than just variations of positives, ICN achieves tighter and more precise decision boundaries. There is zero overhead during inference, making it a practical enhancement for existing architectures.
However, a challenge arises when pseudo-negatives become too realistic, potentially confusing the model if they become indistinguishable from true positives. Future work will explore more sophisticated ways to utilize these high-fidelity late-stage samples rather than simply stopping the training process.
Summary updated