Learning Equivariant Segmentation with Instance-Unique Querying

Wenguan Wang, James Liang, Dongfang Liu

TL;DR

This paper introduces a novel training framework that significantly boosts query-based instance segmentation models by enforcing two key properties: dataset-level uniqueness and transformation equivariance. By forcing models to distinguish instances across the entire dataset (not just within a single image) and ensuring queries adapt predictably to geometric changes, the authors achieve performance gains of up to 3.2 AP on COCO without any architectural changes or inference-time overhead.

What Problem Does This Paper Address?

Modern instance segmentation—the task of identifying and masking every individual object in an image—has shifted toward "query-based" architectures (like Mask2Former or SOLOv2). These models use learnable "query" vectors to search through image features and pull out specific object masks.

However, the authors identify a major limitation in how these models are trained: they are typically only taught to distinguish objects within a single scene. Because a single training image usually contains only a few objects, the model doesn't have to work very hard to create "unique" queries. This "lazy" learning leads to queries that aren't discriminative enough when faced with complex, real-world data. Furthermore, these models often struggle with geometric consistency; if you flip or crop an image, the relationship between the query and the object feature might break.

Methods

The authors propose a "plug-and-play" training framework that can be added to existing query-based models. It consists of two main components designed to enforce more rigorous learning constraints.

Training Framework Overview
  1. Inter-Scene Instance Disambiguation: Instead of just matching a query to pixels in its own image, the model is forced to "mismatch" pixels from other images in the dataset. To make this computationally feasible, the authors maintain an external memory bank of 100,000 pixel samples from across the dataset. This forces the query embeddings to become much more specific and discriminative at a global level.
  2. Transformation Equivariance Constraint: Standard data augmentation teaches a model to be invariant (the output shouldn't change). However, segmentation is inherently equivariant—if you flip the input image, the mask and the features should flip accordingly. The authors introduce a loss that explicitly encourages the queries and features to change in a predictable, synchronized way when geometric transformations (like cropping or flipping) are applied.

To keep training efficient, they use sparse, instance-balanced sampling, randomly picking a fixed number of pixels (e.g., 50) from each object instance to store in the memory bank, ensuring small objects aren't ignored.

Datasets

The researchers evaluated their framework on two gold-standard benchmarks:

  • MS COCO: The standard benchmark for instance segmentation with 80 object categories.
  • LVIS v1: A more challenging "long-tailed" dataset with over 1,200 categories, requiring the model to recognize many rare objects.

Key Insights and Ideas

  • Global vs. Local Discrimination: The "aha moment" here is realizing that intra-scene discrimination is too easy. By treating the entire dataset as a retrieval problem, the model learns "instance-unique" signatures that are far more robust.
  • Equivariance > Invariance: While many models aim for transformation invariance, the authors argue that for spatial tasks like segmentation, equivariance is the more natural and powerful constraint. It ensures the "handshake" between a query and an image feature remains stable even as the object moves or changes scale.
  • Zero-Cost Inference: Because these constraints are applied only during the loss calculation in training, the resulting model is more accurate but runs at the exact same speed as the original "base" model during deployment.

Key Results

The framework was tested on four major models (CondInst, SOLOv2, SOTR, and Mask2Former) with various backbones (ResNet, Swin Transformer).

Performance vs. Inference Speed
  • COCO Gains: The method provided consistent improvements across the board, ranging from +1.6 to +3.2 AP. As shown in the performance chart, these gains come with zero inference speed penalty.
  • New State-of-the-Art: When applied to Mask2Former with a Swin-L backbone, it achieved a record-breaking 51.8 AP on COCO.
  • LVIS Improvement: On the difficult LVIS dataset, it boosted SOLOv2 by 2.7 AP, showing particular strength in improving the masks of rare objects.
  • Qualitative Quality: Visual results show the model is much better at distinguishing between "huddled" objects of the same class (e.g., a crowd of people) where standard models often bleed masks together.

Limitations and Future Work

  • Training Overhead: While there is no cost at inference time, the training process slows down slightly (about 5%) due to the management of the external memory bank and the extra loss calculations.
  • Memory Management: The current implementation uses a fixed-size queue for the memory bank. Future work could explore more sophisticated ways to select "hard negatives" (the most confusing pixels) to store in memory.
  • Generalization: The authors suggest this logic could be extended beyond instance segmentation to other dense prediction tasks like object detection or panoptic segmentation.

Summary updated

Abstract

Prevalent state-of-the-art instance segmentation methods fall into a query-based scheme, in which instance masks are derived by querying the image feature using a set of instance-aware embeddings. In this work, we devise a new training framework that boosts query-based models through discriminative query embedding learning. It explores two essential properties, namely dataset-level uniqueness and transformation equivariance, of the relation between queries and instances. First, our algorithm uses the queries to retrieve the corresponding instances from the whole training dataset, instead of only searching within individual scenes. As querying instances across scenes is more challenging, the segmenters are forced to learn more discriminative queries for effective instance separation. Second, our algorithm encourages both image (instance) representations and queries to be equivariant against geometric transformations, leading to more robust, instance-query matching. On top of four famous, query-based models (i.e., CondInst, SOLOv2, SOTR, and Mask2Former), our training algorithm provides significant performance gains (e.g., +1.6 – 3.2 AP) on COCO dataset. In addition, our algorithm promotes the performance of SOLOv2 by 2.7 AP, on LVISv1 dataset.
BibTeX
@inproceedings{wang2022learning,
  author = {Wenguan Wang and James Liang and Dongfang Liu},
  title = {{Learning Equivariant Segmentation with Instance-Unique Querying}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=e824zXt3G1},
}
Figure