InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling

Xiaoxiao Jiang, Suyi Li, Lingyun Yang, Tianyu Feng, Zhipeng Di, Weiyi Lu, Guoxuan Zhu, Xiu Lin, Kan Liu, Yinghao Yu, Tao Lan, Guodong Yang, Lin Qu, Liping Zhang, Wei Wang

Introduction

Generative image editing, encompassing tasks like virtual try-on and face swapping, has become a cornerstone of AI cloud workloads. Despite its popularity, the underlying diffusion models are computationally intensive. Current serving systems often treat editing an image as a full generation task, ignoring the fact that large portions of the image (the "unmasked" regions) remain unchanged. This leads to a "high-cost, high-latency" bottleneck, as standard attention mechanisms require processing every pixel to maintain global context.

Technical Innovations

InstGenIE addresses these inefficiencies through three core technical pillars designed to exploit spatial sparsity without sacrificing image quality.

1. Mask-aware Caching and Scheduling

The system identifies that activations for unmasked regions are highly redundant across different editing requests for the same template. InstGenIE caches these intermediate activations. During inference, the GPU computes only the masked tokens and "replenishes" the remaining context from the cache. To manage varying computational loads, a regression-based load balancer routes requests based on mask size to minimize overall latency.

2. Bubble-free Pipeline

Because cached activations are too large for GPU memory (HBM), they are stored in host RAM (DRAM). InstGenIE utilizes a Dynamic Programming (DP) algorithm to schedule a pipeline that overlaps the loading of these activations with the computation of masked tokens. This effectively hides transfer latency, preventing "bubbles" in the processing pipeline.

The InstGenIE editing pipeline: from template and mask to high-fidelity output.

3. Disaggregated Continuous Batching

Borrowing from Large Language Model (LLM) serving, InstGenIE implements continuous batching, allowing new requests to join a running batch after any single denoising step. To prevent CPU-bound tasks like image encoding from stalling the GPU, these processes are disaggregated into separate execution environments.

Evaluation and Results

The authors evaluated InstGenIE using production-scale traces (including a 14-day workload from a face-swap service) and benchmarks across models like SD2.1, SDXL, and Flux.

  • Throughput and Latency: InstGenIE achieves up to 3× higher throughput and reduces average request latency by up to 14.7× compared to baselines like Diffusers and TeaCache.
  • Inference Speedup: On the SDXL model using an H800 GPU, the system delivered a 1.7× model inference speedup.
  • Image Quality: The system maintains near-perfect fidelity, achieving SSIM scores of 0.99. It also demonstrated superior text-image alignment (CLIP scores) compared to existing sparse-inference methods.
  • Tail Latency: The combination of continuous batching and disaggregation reduced P95 tail latency by 29%.

Discussion and Future Work

While InstGenIE provides significant gains for masked edits, its benefits are limited for global transformations like style transfer where every pixel changes. Additionally, extremely high-traffic services must manage host memory effectively to avoid cache misses. Future research may extend these mask-aware techniques to video editing, where temporal redundancy between frames offers further opportunities for computational savings.

Summary updated

Abstract

Generative image editing using diffusion models has become a prevalent application in today's AI cloud services. In production environments, image editing typically involves a mask that specifies the regions of an image template to be edited. The use of masks provides direct control over the editing process and introduces sparsity in the model inference. In this paper, we present InstGenIE, a system that efficiently serves image editing requests. The key insight behind InstGenIE is that image editing only modifies the masked regions of image templates while preserving the original content in the unmasked areas. Driven by this insight, InstGenIE judiciously skips redundant computations associated with the unmasked areas by reusing cached intermediate activations from previous inferences. To mitigate the high cache loading overhead, InstGenIE employs a bubble-free pipeline scheme that overlaps computation with cache loading. Additionally, to reduce queuing latency in online serving while improving the GPU utilization, InstGenIE proposes a novel continuous batching strategy for diffusion model serving, allowing newly arrived requests to join the running batch in just one step of denoising computation, without waiting for the entire batch to complete. As heterogeneous masks induce imbalanced loads, InstGenIE also develops a load balancing strategy that takes into account the loads of both computation and cache loading. Collectively, InstGenIE outperforms state-of-the-art diffusion serving systems for image editing, achieving up to 3x higher throughput and reducing average request latency by up to 14.7x while ensuring image quality.
BibTeX
@misc{jiang2025instgenie,
  author = {Xiaoxiao Jiang and Suyi Li and Lingyun Yang and Tianyu Feng and Zhipeng Di and Weiyi Lu and Guoxuan Zhu and Xiu Lin and Kan Liu and Yinghao Yu and Tao Lan and Guodong Yang and Lin Qu and Liping Zhang and Wei Wang},
  title = {{InstGenIE: Generative Image Editing Made Efficient with Mask-aware Caching and Scheduling}},
  year = {2025},
  eprint = {2505.20600},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.20600},
}
Figure