Symmetry Detection with Patch Embedding Models by Means of a Scalar Product Transform

Christoph Dalitz, Jan BeekerarXiv 2026 · arXiv

TL;DR

Researchers have developed the Scalar Product Transform (SPT), a zero-shot method for detecting point reflection symmetry in images by leveraging pre-trained vision transformer (ViT) embeddings. By comparing the feature vectors of mirrored image patches, the SPT outperforms specialized deep learning models and classical gradient-based methods without requiring any task-specific training.

What Problem Does This Paper Address?

Symmetry is a fundamental aspect of human perception, yet teaching machines to detect it reliably remains a challenge. Traditional "rule-based" algorithms often rely on pixel-level gradients, which are sensitive to noise and slight perspective distortions. On the other hand, modern deep learning approaches—like convolutional neural networks (CNNs) specifically trained for symmetry—require large, annotated datasets and often struggle to generalize to objects of varying sizes or those seen under significant skew.

The authors identify a gap: can we use the rich, high-level features already learned by "foundation" vision models (like DINOv3 or Franca) to detect symmetry? These models are trained on massive datasets for general tasks, but the authors hypothesize that their internal representations (embeddings) inherently capture the structural information needed for symmetry detection.

Methods

The core innovation is the Scalar Product Transform (SPT). It is heavily inspired by the classical Gradient Product Transform (GPT), which looks for "anti-parallel" gradients at mirrored positions.

Instead of gradients, the SPT uses patch embeddings. Here is the workflow:

  1. Patching & Embedding: An image is divided into a grid of patches. A pre-trained ViT (like DINOv3) generates a high-dimensional vector (embedding) for each patch.
  2. Handling Rotation: For point reflection symmetry, mirrored patches are rotated by 180° ($\pi$) relative to each other. To account for this, the authors compute two embeddings for every patch: the original ($\vec{A}$) and one from the same patch rotated by $\pi$ ($\vec{B}$).
  3. The Transform: For any candidate symmetry center, the algorithm calculates the scalar product between embeddings of patches at mirrored positions. A high sum of these products indicates strong symmetry.
  4. Post-processing (The "Aha" Moment): Raw embeddings often suffer from "anisotropy"—they are all too similar in angular distance. This causes high symmetry scores even in boring, homogeneous regions (like a flat wall). To fix this, the authors use an "all-but-the-top" approach: they center the embeddings and subtract the three most dominant PCA components. This dramatically cleans up the results.
Figure 2: The Scalar Product Transform (SPT) sums the scalar products of a vector embedding ⃗A with an embedding ⃗B belonging to an opposite patch that has been rotated by π.

Datasets

The authors evaluated their method using the VISAPP dataset (Dalitz et al., 2013). This benchmark contains 159 images of 27 different objects, captured from various perspectives and in different contexts (e.g., "detail" shots vs. "context" shots where the object is small). This variety makes it an excellent test for robustness against skew and background noise.

Key Insights and Ideas

  • Foundation Models are Symmetry-Aware: You don't need to train a model for symmetry. The features learned by self-supervised models like DINOv3 are robust enough to detect geometric properties like point reflection out of the box.
  • Anisotropy is the Enemy: Without post-processing, the embeddings are "too close" to each other, leading to false positives in plain regions. Subtracting the top PCA components (Algorithm 1 in the paper) is essential for making the symmetry signal stand out.
  • Square is Better than Rectangular: While the math allows for rectangular symmetry regions, the authors found that restricting the search to square regions ($r_x = r_y$) actually improved the recognition rate. It effectively filters out "accidental" approximate symmetries that humans don't perceive as meaningful.
  • Implicit Bounding Boxes: Unlike many CNN-based detectors that only output a heatmap of symmetry centers, the SPT naturally yields a bounding box for the symmetric region by finding the radii that maximize the normalized symmetry score.
Figure 3: Example of the effect of the post-processing with Algorithm 1 on the SPT applied to DINOv3 ViT-S. The symmetry scores have been computed with α = 0.5, a sub-patch resolution of 4 pixels, and the restriction rx = ry. For visualization purposes, the image was converted to grayscale and the scores have been overlaid as as a transparent heatmap.

Key Results

The SPT, particularly when using the Franca ViT-L model, set a new bar for performance on the VISAPP dataset:

  • Recognition Rate: Franca-L achieved a 76% total recognition rate, significantly outperforming the state-of-the-art CNN EquiSym (63%) and the classical GPT (46%).
  • Robustness to Skew: The SPT proved exceptionally robust to perspective distortion. In "strong skew" scenarios, Franca-L maintained a 71% success rate, while EquiSym dropped to 56%.
  • Model Comparison: DINOv3 performed better on "detail" images (likely due to a positional bias toward the center), while Franca was superior for "context" images where the symmetric object might be off-center.

Limitations and Future Work

The primary trade-off for the SPT's accuracy is runtime. While EquiSym can process an image in about 0.3 seconds, the SPT takes between 1.6s (DINOv3 ViT-S) and 13.5s (DINOv3 ViT-H+). This makes it less suitable for real-time applications without further optimization or parallelization.

Additionally, the current study focuses exclusively on point reflection ($C_{2m}$) symmetry. Extending the Scalar Product Transform to detect other types of symmetry, such as bilateral reflection or higher-order rotations, remains an open and promising direction for future research. Finally, testing on larger, more diverse datasets will be necessary to confirm the generalizability of these findings.

Summary updated

Abstract

We present a transformation of vector embeddings representing image patches into scores for point reflection symmetry at the crossing points of patches. The transformation is heavily inspired by the Gradient Product Transform, but it uses the vector embeddings instead of the grayscale gradients. We therefore call it the "Scalar Product Transform" (SPT). Unlike deep learning models for symmetry detection, the method does not require a specific training step tailored for symmetry detection, but it works with arbitrary pre-trained patch embedding models. We have tested the method with DINOv3 and Franca. In our tests, the method outperformed the Gradient Product Transform and was even better than EquiSym, a convolutional neural network specifically designed and trained for symmetry detection. Although the runtime of the SPT is greater than that of EquiSym, the SPT has the advantage that it directly yields a bounding box for the symmetric region.
BibTeX
@misc{dalitz2026symmetry,
  author = {Christoph Dalitz and Jan Beeker},
  title = {{Symmetry Detection with Patch Embedding Models by Means of a Scalar Product Transform}},
  year = {2026},
  eprint = {2611.21363},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21363},
}
Figure