Introduction
Generative Adversarial Networks (GANs), particularly StyleGAN2, have achieved remarkable success in synthesizing high-quality, photo-realistic images. However, "steering" these models remains a significant challenge due to the entangled nature of their latent spaces. In complex scenes like bedrooms, attempting to change a single element—such as the color of a bed—often inadvertently alters unrelated parts of the image, like the wallpaper or window placement. This global entanglement prevents the fine-grained control necessary for practical design applications.
LoGAN (Local GAN) introduces a framework for precise, local control over image generation. By shifting manipulation from global latent codes to intermediate feature maps, LoGAN enables users to "re-decorate" rooms by removing, inserting, shifting, and rotating individual objects while maintaining seamless integration and perspective. This approach treats a generated image not as a flat grid of pixels, but as a collection of modifiable parts.
Methodology: Local Feature Manipulation
The core innovation of LoGAN lies in its use of intermediate feature maps, which encode rich spatial information. The framework employs several key components to achieve local control:
- Content and Style Modulation (CMod & SMod): These operators allow for replacing features in specific masked regions or applying style transformations (via AdaIN) to local objects without affecting the global scene. This allows a user to change the texture of a bed without altering the rest of the room.
- Priority Masking: To handle depth and occlusion (e.g., a lamp positioned in front of a window), a priority system determines which object's features "win" in the final composition.
- Layout Parsing: The system predicts basic room geometry (walls, floor, ceiling) using a semantic segmentation model to create a "clean slate" for editing.
Key Insights: Layer Sensitivity
A critical finding of the LoGAN research is that different layers within the GAN generator are specialized for different types of edits. The researchers discovered that:
- Removal is most effective at early layers (e.g., Layer 4). Because these layers have a larger "receptive field," the generator can better "hallucinate" the background to fill the hole left by a removed object.
- Insertion is best performed at middle layers (e.g., Layer 7). This preserves the sharp details and textures of the new object without it being "eroded" by the surrounding background features.
By successfully generating a plausible empty room based on the original's geometry, the authors provide a stable foundation for complex multi-object editing that wasn't possible with previous global latent space methods.
Results and Multi-Object Composition
LoGAN demonstrates impressive qualitative results in seamless editing and multi-object composition. The system can remove a bed and replace it with a different one, or move a window across a wall, without creating visible "seams" or artifacts. Users can progressively build a room from scratch, adding furniture and decor that respect the room's perspective and lighting.
The system maintains style consistency across edits, ensuring that newly inserted objects match the existing lighting and aesthetic of the room. This level of control transforms the generative process from a "black box" into an interactive design tool.
Limitations and Future Work
Despite its strengths, LoGAN relies on the accuracy of external semantic segmentation models for layout parsing. Additionally, current object insertion requires "source" objects from the GAN's existing latent space rather than external user-provided images. Future research aims to extend this local control to other complex environments, such as kitchens or living rooms, and integrate these capabilities into user-friendly interior design software.
Summary updated