TL;DR
This paper introduces Convolution Sliced Wasserstein (CSW), a novel approach to measuring distances between image distributions. By replacing traditional vectorization and matrix multiplication with convolution operators, CSW preserves critical spatial structures and significantly reduces memory overhead, leading to faster convergence and higher-quality results in deep generative modeling.
The Problem with Standard Sliced Wasserstein on Images
The Wasserstein distance is a powerful tool for comparing probability distributions, but it is notoriously expensive to compute and suffers from the "curse of dimensionality." Sliced Wasserstein (SW) distances offer a workaround by projecting high-dimensional distributions onto one-dimensional lines, where the distance is easy to calculate.
However, when applied to images, standard SW has a major flaw: it requires vectorization. To "slice" an image, you must first flatten it into a long vector, destroying the 2D spatial relationships (like edges and textures) that define the image. Furthermore, in high-resolution settings, storing these projection vectors becomes a memory nightmare—a single projection for a $128 \times 128$ image requires a vector of the exact same size. This paper aims to bridge the gap between the efficiency of slicing and the inherent spatial nature of image data.
Methods: The Convolution Slicer
To address the limitations of vectorization, the authors propose replacing the standard Radon transform (matrix multiplication) with a Convolution Slicer.
Instead of one giant projection vector, CSW uses a sequence of small, learnable (or randomly sampled) convolution kernels to progressively map an image down to a scalar value. They introduce three main variants of the CSW:
- CSW-base (CSW-b): Uses standard convolutions to reduce image size.
- CSW-stride (CSW-s): Incorporates strided convolutions to downsample more efficiently.
- CSW-dilation (CSW-d): Uses dilated convolutions to increase the receptive field, allowing the slicer to capture global information without increasing the number of parameters.
The authors also explore Non-linear CSW, which adds activation functions between convolution layers to capture more complex, non-linear features of the data. Mathematically, they prove that CSW is a pseudo-metric and maintains the same favorable sample complexity as standard SW, meaning it avoids the curse of dimensionality.
Key Insights and Results
The researchers evaluated their methods on standard computer vision benchmarks, including MNIST, CIFAR10 ($32 \times 32$), CelebA ($64 \times 64$), CelebA-HQ ($128 \times 128$), and STL10 ($96 \times 96$).
- Spatial Awareness is Key: By using convolutions, the "slices" are forced to respect local pixel correlations. This makes the resulting one-dimensional projections more "discriminative," meaning they are better at catching the subtle differences between a real image and a blurry generated one.
- The Memory Win: Standard SW memory scales with the total number of pixels ($O(cd^2)$). In contrast, the projection memory for CSW-stride and CSW-dilation scales much more favorably ($O(c + \log d)$). This makes CSW far more scalable for high-resolution image generation.
- Better Image Quality & Faster Convergence: When integrated into a generative modeling framework, CSW outperformed vanilla SW. On CelebA-HQ ($128 \times 128$), CSW-s achieved a significantly lower FID score (a measure of image realism) of 22.25 compared to SW's 28.19 when using 1,000 projections. CSW variants also generally helped generative models converge in fewer epochs.
- Performance with Fewer Slices: One of the most striking results was that CSW performs remarkably well even with very few projections ($L=1$). On CelebA, CSW-s with $L=1$ outperformed vanilla SW with $L=100$.
Limitations and Future Work
While CSW is highly effective for images, its current formulation is specifically tailored for grid-like data. Extending this "convolutional slicing" logic to other structured data, like graphs or point clouds, remains an open challenge.
Additionally, while the memory complexity is lower, the computational complexity of the "base" version (CSW-b) is actually higher than standard SW. While the stride and dilation versions mitigate this, there is still room to optimize the implementation for even faster training. The authors also suggest that further theoretical work could investigate the conditions under which CSW becomes a full metric rather than a pseudo-metric.
Summary updated