Abstract
Recent Image-to-Image Translation (IIT) algorithms have achieved significant progress in neural style transfer and image attribute manipulation. However, existing approaches typically require exhaustively labeled training data, which is labor-demanding and difficult to scale. To overcome this limitation, we propose Sparsely Grouped Generative Adversarial Networks (SG-GAN), a novel approach capable of translating images on sparsely grouped datasets where only a small fraction of samples are labeled. Utilizing a one-input multi-output architecture, SG-GAN is uniquely suited for sparsely grouped and multi-task learning, allowing image translation across multiple groups within a single model. Experimental results on facial attribute manipulation tasks demonstrate that SG-GAN produces results comparable to state-of-the-art methods on fully labeled datasets and superior quality on sparsely grouped datasets.
Introduction: The Challenge of Labeling in Image Translation
Image-to-Image Translation has seen massive leaps recently, enabling complex edits such as changing hair color, adding smiles, or swapping genders in photographs. However, most existing models suffer from a significant "labeling tax."
Supervised methods require perfectly paired images, which are nearly impossible to collect at scale. Unsupervised methods, such as CycleGAN or StarGAN, use "grouped" data—requiring separate sets of images for each attribute. While easier than paired data, this still necessitates exhaustive labeling of every image in a dataset. For large-scale datasets like CelebA, which contains over 200,000 images across 40 attributes, the manual labor required is staggering.
As illustrated in Figure 1, SG-GAN addresses this by operating on "mixed" datasets. By leveraging a small minority of labeled samples alongside a large majority of unlabeled data, the model learns the underlying distribution of faces while using sparse labels to guide specific attribute translations.
The SG-GAN Architecture
The proposed SG-GAN architecture introduces three primary innovations to improve data efficiency and translation quality:
- Sparsely Grouped Learning: This semi-supervised approach allows the discriminator to act as both a critic (distinguishing real from fake) and a classifier (identifying attributes). This dual role enables the model to stay grounded in the reality of face structures using unlabeled data, preventing overfitting to the small labeled set.
- One-Input Multi-Output Architecture: Unlike previous models that feed a target label into the generator as an input, SG-GAN’s generator branches out at the end. Each attribute (e.g., gender, hair color) has its own dedicated output branch. This multi-task structure prevents "attribute interference," where changing one feature accidentally alters unrelated features like skin tone or background.
- Refined Residual Image Learning (RRIL): Standard residual learning ($x + G(x)$) can lead to sparse, ghostly artifacts. SG-GAN instead concatenates the input image with feature maps and passes them through an additional convolution layer. This allows the network to more flexibly learn the relationship between original pixels and desired modifications.
Experimental Results and Insights
The researchers evaluated SG-GAN using the CelebA dataset, creating subsets with varying levels of label sparsity (from 100% labels down to only 500 labels per attribute, or roughly 0.5% of the training set).
Label Efficiency
One of the most striking findings was the model's efficiency. SG-GAN(500) achieved a classification accuracy of 93.36% for gender translation using only 500 labels. For comparison, StarGAN—using the full dataset—achieved 96.23%. The quality drop is remarkably small considering SG-GAN used 99% fewer labels.
Background Preservation
Using the MS-SSIM metric to measure background consistency, SG-GAN scored significantly higher (70-72) than StarGAN (64) and ResidualGAN (63). This confirms that the branched architecture successfully protects unrelated parts of the image from "bleeding" effects during translation.
Visual Quality
Qualitatively, SG-GAN produces sharper images with fewer artifacts in hair and skin regions. By separating the decoders for different attributes, the model effectively disentangles features, leading to more realistic textures compared to models that use a single shared output path.
Conclusion and Future Work
SG-GAN represents a major step toward data-efficient image translation. By utilizing unlabeled data and a specialized multi-task architecture, it achieves high-quality results without the prohibitive cost of exhaustive manual annotation.
While the model excels at texture-based changes, future work will address "unnatural geometric translations"—large-scale structural changes like jawline modification—which remain a challenge for current GAN architectures. Additionally, further exploration into semantic correction for highly correlated attributes could further enhance the model's robustness in extremely sparse labeling environments.
Summary updated