Unsupervised Image Decomposition in Vector Layers

Othman Sbai, Camille Couprie, Mathieu Aubry

Abstract

We present a novel unsupervised deep learning paradigm that decomposes images into a sequence of single-color "vector layers." Unlike traditional generative models that output fixed pixel grids, our approach represents images as a stack of parametric transparency masks. This method enables intuitive image editing, high-quality vectorization, and semantic image search without the need for manual labels.

Introduction

Modern deep image generation models, such as GANs, produce high-fidelity results but often operate as "black boxes." Modifying specific attributes—like hair color or object shape—typically requires expensive manual labeling or handcrafted tools. Furthermore, standard neural networks are constrained by fixed resolutions, leading to blurring upon magnification. This research addresses these challenges of interpretability and scalability by mimicking a human designer's workflow: constructing an image layer-by-layer using primitive shapes and colors.

Methodology

The proposed Layered Vector Image Generation framework utilizes a recursive architecture to build images iteratively.

Recursive Architecture

At each step $t$, a Residual Network (ResNet) analyzes the current canvas and the target image to predict a color ($c_t$) and a set of shape parameters ($p_t$).

The Mask Generator (MLP)

The shape parameters are processed by a Multi-Layer Perceptron (MLP) that functions as a coordinate-based mask generator. For any given $(x, y)$ coordinate, the MLP outputs a transparency value (alpha mask). Because this is a continuous mathematical function, the mask can be rendered at any resolution, providing a true vector representation.

An emergent hair mask layer generated by the MLP-based mask generator without any manual supervision.

Alpha Blending and Training

The colored masks are blended onto the canvas sequentially. The model is trained in an unsupervised manner by minimizing the difference between the reconstruction and the target image using $L_1$ loss, perceptual loss (VGG16), and optional adversarial loss.

Experiments and Results

The model was evaluated on CelebA, ImageNet, and MNIST datasets.

Original target image from the CelebA dataset used for unsupervised training and decomposition.

Quantitative and Qualitative Performance

On the CelebA dataset, the model achieved a PSNR of approximately 24.02. While competitive with standard autoencoders, the primary advantage lies in editability. Users can interact with specific masks (e.g., lips or hair) to modify color or shape instantaneously.

Emergent Semantics

A key finding is the emergence of semantic decomposition. Without explicit labels, the model naturally learns to assign specific layers to features like hair, skin, or accessories, as this is the most efficient reconstruction strategy. This allows for attribute retrieval; for instance, selecting a "hat" mask from one image allows the system to find other images with hats purely through unsupervised decomposition.

Discussion and Limitations

The recursive approach significantly outperforms "one-shot" methods, as the model benefits from seeing the current state of the canvas before adding the next layer. The use of parametric functions allows for training at low resolutions (e.g., $128 \times 128$) while rendering at high resolutions (e.g., $1024 \times 1024$) without pixelation.

However, limitations remain. The computational cost of recursive rendering is higher than one-shot models. Additionally, the "single color" constraint per layer makes representing complex gradients or textures difficult within a single primitive. Future work may extend this framework to video or integrate it into generative adversarial networks for high-resolution vector art creation.

Summary updated

Abstract

Deep image generation is becoming a tool to enhance artists and designers creativity potential. In this paper, we make the generation process more structured and easier to interact with. We propose a new deep image reconstruction paradigm where the outputs are composed from simple layers, defined by their color and a vector transparency mask. This presents a number of advantages compared to the commonly used convolutional network architectures. In particular, our layered decomposition allows simple user interaction, for example to update a given mask, or change the color of a selected layer. From a compact code, our architecture also generates vector images with a virtually infinite resolution, the color at each point in an image being a parametric function of its coordinates. We validate the efficiency of our approach by comparing reconstructions with state-of-the-art baselines given similar memory resources on CelebA and ImageNet datasets. We demonstrate several applications of our new image representation obtained in an unsupervised manner, including editing, vectorization and image search.
BibTeX
@inproceedings{sbai2020unsupervised,
  author = {Othman Sbai and Camille Couprie and Mathieu Aubry},
  title = {{Unsupervised Image Decomposition in Vector Layers}},
  booktitle = {ICIP},
  year = {2020},
  doi = {10.1109/icip40778.2020.9190638},
}
Figure