Abstract
MixerGAN is a novel architecture for unpaired image-to-image translation that replaces traditional convolutional and transformer layers with MLP-Mixer blocks. It aims to capture long-range spatial dependencies more efficiently than attention-based Transformers, which suffer from quadratic memory scaling. By utilizing token-mixing and channel-mixing MLPs, MixerGAN achieves competitive visual results on standard benchmarks like Apple-to-Orange and Yosemite seasonal translation, while maintaining linear memory complexity relative to the number of image patches.
Introduction
The field of computer vision has long been dominated by two primary architectures: Convolutional Neural Networks (CNNs) and Transformers. While CNNs are computationally efficient, their inherent "locality bias" limits their ability to model long-range dependencies between distant pixels. Transformers address this through self-attention mechanisms, but at a significant cost: memory usage scales quadratically ($O(n^2)$) with the number of pixels, making high-resolution generation difficult.
MixerGAN proposes a "middle ground" for unpaired image-to-image translation. The goal is to capture global context without the prohibitive memory overhead of Transformers. This is particularly relevant for tasks where direct "before and after" training pairs are unavailable, requiring the model to learn underlying distributions.
Methods
MixerGAN is built upon the CycleGAN framework, utilizing two generators and two discriminators with cycle-consistency loss. The key innovation lies in the generator's bottleneck, where standard residual convolutional blocks are replaced with MLP-Mixer blocks.
The MLP-Mixer Block
The architecture employs two distinct types of Multi-Layer Perceptrons:
- Token-mixing MLPs: These operate across spatial locations, allowing the model to capture global context and long-range communication.
- Channel-mixing MLPs: These operate across the feature dimension at each spatial location, enabling communication between different feature maps.
To process images, the model uses Patch Projection, linearly projecting $8 \times 8$ image patches into tokens. This reduces the sequence length, ensuring that MLP operations remain computationally manageable. The training is further stabilized using the Least Squares GAN (LS-GAN) formulation.
Experimental Results
The authors evaluated MixerGAN on three standard unpaired translation datasets: Apple $\leftrightarrow$ Orange, Horse $\leftrightarrow$ Zebra, and Yosemite Summer $\leftrightarrow$ Winter.
Quantitative Performance
MixerGAN demonstrated strong performance as measured by the Kernel Inception Distance (KID). Notably, on the Orange-to-Apple task, MixerGAN achieved a KID of 4.50, significantly outperforming the original CycleGAN's score of 9.82.
Qualitative Analysis
Qualitatively, MixerGAN often produces more coherent and vibrant results than its convolutional counterparts. In the Yosemite dataset, it generated more realistic snow and vivid skies compared to the muted tones often seen in CycleGAN outputs.
Efficiency
A key advantage of MixerGAN is its memory efficiency. Unlike self-attention mechanisms that explode in memory usage as resolution increases, the Mixer blocks exhibit relatively flat memory growth, confirming their suitability for higher-resolution tasks.
Discussion and Limitations
Despite its successes, MixerGAN faces challenges related to its patch-based processing. The use of $8 \times 8$ patches can occasionally lead to visible "seams" or repetitive artifacts in the output. Furthermore, the aggressive down-projection of patches into tokens represents a significant information bottleneck, which may limit the generator's capacity for highly detailed translations. Future work may involve increasing latent channel depth or integrating contrastive learning to further refine image quality.
Summary updated