Abstract
Semi-supervised generative learning (SSGL) makes use of unlabeled data to achieve a trade-off between the data collection/annotation effort and generation performance, when adequate labeled data are not available. Learning precise class semantics is crucial for class-conditional image synthesis with limited supervision. Toward this end, we propose a semi-supervised Generative Adversarial Network with a Mask-Embedded Discriminator, which is referred to as MED-GAN. By incorporating a mask embedding module, the discriminator features are associated with spatial information, such that the focus of the discriminator can be limited in the specified regions when distinguishing between real and synthesized images. A generator is enforced to synthesize the instances holding more precise class semantics in order to deceive the enhanced discriminator. Also benefiting from mask embedding, region-based semantic regularization is imposed on the discriminator feature space, and the degree of separation between real and fake classes and among object categories can thus be increased. This eventually improves class-conditional distribution matching between real and synthesized data. In the experiments, the superior performance of MED-GAN demonstrates the effectiveness of mask embedding and associated regularizers in facilitating SSGL.
Introduction
Class-conditional image synthesis—the task of generating images belonging to specific categories—typically requires vast amounts of labeled data. In many practical scenarios, obtaining such labels is prohibitively expensive or requires expert domain knowledge. Semi-supervised generative learning (SSGL) addresses this by leveraging a small set of labeled data alongside a larger pool of unlabeled samples.
A primary obstacle in SSGL is the "Discriminator Shortcut." Standard discriminators often learn to distinguish real from fake images by focusing on easily identifiable global statistics or background textures rather than the complex semantic features of the target objects. This issue is exacerbated when labels are scarce, leading to generators that fail to capture fine-grained class details. This paper introduces MED-GAN, a framework designed to overcome these shortcuts through spatial guidance.
The MED-GAN Framework
MED-GAN consists of three interacting components: a Generator ($G$), a Discriminator ($D$), and a Classifier ($C$). The core innovation lies in the Mask-Embedded Discriminator, which forces the model to focus on specific image regions rather than global shortcuts.
The framework employs a technique called random regional replacement, where a rectangular patch from a real image is pasted onto a synthesized image (or vice versa). To prevent the discriminator from becoming confused by these composite images, a binary mask ($m$)—indicating the exact location of the replacement—is fed into a dedicated embedding module ($A_{mask}$). This provides the discriminator with spatial awareness, allowing it to judge specific pixels as "real" or "fake" based on the provided mask.
To further refine the discriminator's feature space, the authors introduce region-based semantic regularization:
- Consistency Loss ($L_{cons}$): Ensures internal features remain stable despite variations in mask geometry.
- Contrastive Loss ($L_{ctrs}$): Pushes the features of "real" regions closer to original real images in the feature space while distancing them from "fake" regions.
Experimental Results
MED-GAN was evaluated on standard benchmarks including CIFAR-10/100, FaceScrub (FS-100), and the fine-grained CUB-200 bird dataset. The model consistently outperformed previous state-of-the-art (SOTA) methods, particularly on complex datasets where semantic precision is vital.
On the challenging CUB-200 dataset, MED-GAN achieved a Frechet Inception Distance (FID) of 16.90, a massive improvement over the previous SOTA, $R^3$-CGAN, which recorded an FID of 88.62. On CIFAR-10, the model improved the Inception Score (IS) to 8.47 and reduced the classification test error rate to 6.02%, demonstrating that high-quality synthesis and robust classification can be mutually beneficial in a joint training regime.
Conclusion
By explicitly embedding spatial masks into the discriminator and applying region-based semantic regularization, MED-GAN effectively mitigates the discriminator shortcut. This approach enables the learning of precise class semantics under limited supervision, significantly advancing the state-of-the-art in semi-supervised class-conditional image synthesis.
Summary updated