Symbolic Disentangled Representations for Images

Alexandr Korchemnyi, Alexey K. Kovalev, Aleksandr I. PanovarXiv 2024 · DOI · arXiv

Introduction

In representation learning, "disentanglement" refers to the ability of a model to isolate independent generative factors of an image—such as color, shape, and position—into distinct components of its latent space. Traditional approaches, like $\beta$-VAEs, typically rely on localist representations, where individual coordinates in a latent vector are mapped to specific properties. However, these representations often lack robustness and become increasingly difficult to interpret as dimensionality grows.

ArSyD (Architecture for Symbolic Disentanglement) introduces a shift toward distributed representations by leveraging Hyperdimensional Computing (HDC). Instead of single numerical values, ArSyD represents generative factors as high-dimensional "symbolic" hypervectors, enabling more robust, noise-resistant, and interpretable image manipulation through algebraic operations.

Methodology: Symbolic Disentanglement

The core innovation of ArSyD lies in its use of Vector Symbolic Architectures (VSA). In this framework, properties are not just indices but full-sized vectors that can be combined via superposition.

The ArSyD Architecture

The model consists of an encoder that extracts features from an input image, followed by a Generative Factor (GF) Projection module. This module uses an attention mechanism to select property values from a fixed codebook (item memory). These values are then combined using HDC operations:

  • Bundling (Addition): Acts like a set operation, allowing multiple independent properties to coexist in the same vector space.
  • Binding (Circular Convolution): Links specific attributes to their values (e.g., associating the attribute "Color" with the value "Red").
Figure 2: (a) The logic of symbolic representation using item memory and bundling/binding operations. (b) The GF Projection module using attention to interface neural features with symbolic codebooks.

Weakly Supervised Training

ArSyD is trained using a weakly supervised "feature exchange" paradigm. The model is presented with pairs of images that differ by exactly one factor (e.g., two scenes identical except for the object's color). By swapping the symbolic vectors corresponding to the differing factor between the two images, the model learns to isolate and reconstruct properties accurately.

Figure 1: The high-level ArSyD pipeline showing how a target image's properties (e.g., Red) can be exchanged with a donor's properties (e.g., Purple) to generate a reconstructed image with the modified attribute.

Evaluation and Metrics

Because standard disentanglement metrics are designed for coordinate-based vectors, the authors introduced two new metrics: Disentanglement Modularity Metric (DMM) and Disentanglement Compactness Metric (DCM). These metrics evaluate disentanglement by observing how changes in the latent space affect a classifier's prediction in the reconstructed pixel space, making them architecture-agnostic.

Key Results

ArSyD was evaluated on benchmarks including dSprites, CLEVR, and CelebA. Key findings include:

  • High-Quality Reconstruction: On the CLEVR1 dataset, ArSyD achieved an FID score of 93.72, significantly outperforming FactorVAE (115.61) and $\beta$-VAE (129.68).
  • Precise Segmentation: The model reached a 98.37% Intersection over Union (IoU) on dSprites, surpassing standard VAE baselines.
  • Multi-Object Compositionality: When combined with Slot Attention, ArSyD successfully modified specific objects within complex scenes (CLEVR5) without affecting the rest of the image, demonstrating the power of symbolic superposition.

Conclusion and Future Work

ArSyD demonstrates that symbolic hypervectors provide a powerful alternative to traditional latent representations. By treating properties as symbols that can be added or subtracted, the model achieves superior interpretability and editing capabilities. Future research aims to reduce the dependency on paired data and scale the architecture to more complex, non-synthetic datasets by integrating pre-trained feature extractors.

Summary updated

BibTeX
@misc{korchemnyi2024symbolic,
  author = {Alexandr Korchemnyi and Alexey K. Kovalev and Aleksandr I. Panov},
  title = {{Symbolic Disentangled Representations for Images}},
  year = {2024},
  eprint = {2412.19847},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.19847},
}
Figure