Abstract
We present a novel unsupervised deep learning paradigm that decomposes images into a sequence of single-color "vector layers." Unlike traditional generative models that output fixed pixel grids, our approach represents images as a stack of parametric transparency masks. This method enables intuitive image editing, high-quality vectorization, and semantic image search without the need for manual labels.
Introduction
Modern deep image generation models, such as GANs, produce high-fidelity results but often operate as "black boxes." Modifying specific attributes—like hair color or object shape—typically requires expensive manual labeling or handcrafted tools. Furthermore, standard neural networks are constrained by fixed resolutions, leading to blurring upon magnification. This research addresses these challenges of interpretability and scalability by mimicking a human designer's workflow: constructing an image layer-by-layer using primitive shapes and colors.
Methodology
The proposed Layered Vector Image Generation framework utilizes a recursive architecture to build images iteratively.
Recursive Architecture
At each step $t$, a Residual Network (ResNet) analyzes the current canvas and the target image to predict a color ($c_t$) and a set of shape parameters ($p_t$).
The Mask Generator (MLP)
The shape parameters are processed by a Multi-Layer Perceptron (MLP) that functions as a coordinate-based mask generator. For any given $(x, y)$ coordinate, the MLP outputs a transparency value (alpha mask). Because this is a continuous mathematical function, the mask can be rendered at any resolution, providing a true vector representation.
Alpha Blending and Training
The colored masks are blended onto the canvas sequentially. The model is trained in an unsupervised manner by minimizing the difference between the reconstruction and the target image using $L_1$ loss, perceptual loss (VGG16), and optional adversarial loss.
Experiments and Results
The model was evaluated on CelebA, ImageNet, and MNIST datasets.
Quantitative and Qualitative Performance
On the CelebA dataset, the model achieved a PSNR of approximately 24.02. While competitive with standard autoencoders, the primary advantage lies in editability. Users can interact with specific masks (e.g., lips or hair) to modify color or shape instantaneously.
Emergent Semantics
A key finding is the emergence of semantic decomposition. Without explicit labels, the model naturally learns to assign specific layers to features like hair, skin, or accessories, as this is the most efficient reconstruction strategy. This allows for attribute retrieval; for instance, selecting a "hat" mask from one image allows the system to find other images with hats purely through unsupervised decomposition.
Discussion and Limitations
The recursive approach significantly outperforms "one-shot" methods, as the model benefits from seeing the current state of the canvas before adding the next layer. The use of parametric functions allows for training at low resolutions (e.g., $128 \times 128$) while rendering at high resolutions (e.g., $1024 \times 1024$) without pixelation.
However, limitations remain. The computational cost of recursive rendering is higher than one-shot models. Additionally, the "single color" constraint per layer makes representing complex gradients or textures difficult within a single primitive. Future work may extend this framework to video or integrate it into generative adversarial networks for high-resolution vector art creation.
Summary updated