When training Generative Adversarial Networks (GANs) for high-resolution image synthesis, the discriminator faces an inherent architectural conflict: the trade-off between receptive field size and detail resolution.
To ensure that an entire image is structurally coherent—meaning that a face has symmetrically placed eyes, or that the lighting across a landscape is uniform—a discriminator needs a large receptive field. It must look at the "big picture" to enforce global context. However, when a discriminator's receptive field expands to encompass the entire image, it naturally averages out high-frequency details. It becomes blind to micro-textures, sharp boundaries, and fine-grained local features. Conversely, if the discriminator is designed with a small receptive field to focus on fine textures, it loses sight of global geometry, resulting in images that are locally detailed but globally nonsensical.
Dual-scale adversarial discrimination solves this dilemma by decoupling the evaluation of macro-structure from micro-texture. Instead of forcing a single neural network to judge both scales simultaneously, this approach splits the adversarial critique into specialized pathways.
The Architecture of a Two-Tiered Critique
The core mechanism relies on operating multiple discriminators (or distinct discriminator heads) at different spatial scales. In a typical implementation, the generator's output is evaluated by two distinct critics:
- The Global Discriminator: This network looks at the full image (often downsampled to a lower resolution). Its job is to evaluate overall composition, semantic logic, and long-range consistency. It ensures that the generated object fits naturally within its broader environment.
- The Local Discriminator: This network operates on cropped patches, specific regions of interest, or the original high-resolution sub-segments. It is entirely blind to the overall composition and focuses exclusively on high-frequency texture realism, sharp boundaries, and local structural correctness.
┌───► Downsample ───► Global Discriminator ───► Global Loss (Structure)
│
Generated Image ──┤
│
└───► Crop Patch ───► Local Discriminator ───► Local Loss (Texture)
By separating these concerns, the generator receives two distinct, complementary error signals. The global critic penalizes large-scale structural errors, while the local critic penalizes blurriness and micro-artifacts.
Why It Works: Gradient Synergy
To understand why this works, consider the common failure modes of single-scale generation: tiling artifacts, boundary seams, and structural blurriness. Dual-scale discrimination provides explicit gradients to counteract each of these issues.
Eliminating Tiling and Boundary Artifacts
In image editing and composition, a frequent problem is the "boundary artifact," where a newly synthesized patch looks realistic on its own but fails to blend into its surroundings. As shown by Fangneng Zhan et al. in Adversarial Image Composition with Auxiliary Illumination, a local discriminator can explicitly evaluate the transition boundaries and fine details of an inserted patch, while a global discriminator ensures that the composite image maintains overall scene consistency and lighting logic.
Without this dual constraint, generators often produce severe color shifts or repetitive patterns. Xitong Yang et al. demonstrated this in Towards Perceptual Image Dehazing by Physics-Based Disentanglement and Adversarial Training, where utilizing multi-scale discriminators effectively eliminated tiling artifacts while preserving sharp, high-frequency textures across the scene.
Preserving Identity and Anatomical Fidelity
In facial synthesis, the trade-off between global harmony and local sharpness is particularly acute. Junhong Huang et al. showed in Cartoon-to-Photo Facial Translation with Generative Adversarial Networks that a global discriminator ensures overall skin texture consistency, while a local discriminator prevents a "fractured skin" effect by keeping individual anatomical features like eyes and mouths sharp.
When parts of a face are missing or occluded, standard networks tend to output blurry approximations because the global loss averages out the fine details. To combat this, Jiankang Deng et al. introduced a local discriminator focused specifically on central facial features in UV-GAN: Adversarial Facial UV Map Completion for Pose-Invariant Face Recognition. This local critic forces the generator to reconstruct high-fidelity, identity-defining regions with the sharpness necessary for downstream machine recognition tasks, while the global critic maintains the broader head structure.
Enforcing Layout and Object Coherence
The strategy is equally vital when generating complex scenes from spatial layouts (such as bounding boxes). In Image Generation From Layout., Bo Zhao et al. leveraged a global discriminator for overall scene coherence alongside local discriminators for individual object regions. The local discriminators ensure that each constituent object maintains its class-specific details and internal textures, while the global discriminator prevents the objects from looking like disjointed stickers pasted onto a random background.
Generalization Beyond 2D Images
The intuition behind dual-scale discrimination is not limited to standard 2D image pixels; it represents a fundamental principle for stable, high-fidelity generative modeling across diverse modalities.
In a comprehensive review of image expansion techniques, Kyeongbo Kong in A Review on Deep Learning-based Image Outpainting noted that dual-scale architectures are what allow models to maintain both high-level logical consistency and fine-grained textural detail in newly expanded areas.
The concept even translates to 3D structural domains. When generating volumetric data, William Gao et al. in TetGAN: A Convolutional Neural Network for Tetrahedral Mesh Generation employed a global discriminator for overall structural coherence alongside a patch-based discriminator for local surface details. This proved that whether dealing with pixels, voxels, or tetrahedral meshes, splitting the adversarial oversight into macro and micro scales is a highly effective way to achieve both structural integrity and realistic surface textures.
Key Foundational Work
Generative Face Completion (2017) — contributed 2 concept elements to this direction.
Towards Perceptual Image Dehazing by Physics-Based Disentanglement and Adversarial Training (2018) — contributed 2 concept elements to this direction.
Generative Face Completion (2017) — contributed 2 concept elements to this direction.