Competing for Pixels: A Self-Play Algorithm for Weakly-Supervised Semantic Segmentation

Shaheer U. Saeed, Shiqi Huang, João Ramalhinho, Iani J. M. B. Gayo, Nina Montaña Brown, Ester Bonmati, Stephen P. Pereira, Brian R. Davidson, Dean C. Barratt, Matthew J. Clarkson, Yipeng Hu

Introduction

Image segmentation is a cornerstone of computer vision, yet its reliance on dense, pixel-level annotations presents a significant bottleneck, particularly in domains like medical imaging where expert time is at a premium. Weakly-supervised segmentation (WSS) addresses this by training models using only image-level labels (e.g., "this image contains a tumor"). However, WSS faces a persistent "correspondence gap": models often struggle to distinguish between the most discriminative parts of an object (leading to under-segmentation) and the surrounding background (leading to over-segmentation).

This paper introduces RL Self-Play (RLSP), a novel reinforcement learning framework that reframes segmentation as a competitive game. By tasking two agents with "claiming" object-related patches, the system learns to precisely delineate regions of interest (ROIs) without the need for manual masks.

The RLSP Framework

The core of the RLSP approach is the gamification of the segmentation process. The image is divided into a grid of patches, and two agents (Agent A and Agent B) compete to select patches that contain the target object.

1. The Competitive Mechanism

Agents take turns selecting patches. Once a patch is chosen, it is removed from the available pool. A pre-trained "referee"—a binary classifier trained on image-level labels—provides a reward signal by scoring the likelihood that a selected patch contains the object. The agent that selects the higher-scoring patch in a turn wins that round. This competition acts as a natural regularizer: if one agent fails to identify a relevant part of the object, the opponent will claim it to gain a reward, forcing both agents to be more thorough.

2. Termination and the "Showdown"

To ensure the agents are exhaustive, the game includes a strategic termination condition. An agent can choose to end the game at any time. However, if the opponent can still find a patch with a high object-presence score after the game is called, the agent who terminated the game receives a penalty. This "showdown" mechanism forces the agents to learn an optimal stopping strategy, effectively discovering the object's boundaries without hand-tuned probability thresholds.

3. Robust Training via Self-Play

To prevent the models from falling into local optima or cyclic behaviors, the authors employ Prioritized Fictitious Self-Play. Instead of simply playing against the most recent version of itself, the current agent competes against a distribution of its past iterations. This ensures the learned strategies are robust and generalize well across different image contexts.

Experimental Results

The RLSP method was evaluated across four diverse datasets, spanning standard computer vision benchmarks and complex medical imaging tasks:

  • PASCAL VOC 2012 & MS COCO 2014: On PASCAL VOC, the model achieved 78.9 mIoU, surpassing the previous state-of-the-art (MARS) by 1.2 points.
  • Medical Imaging (CMPS & LiTS): In the Male Pelvic Structures (CMPS) dataset, RLSP reached a 58.8 mDSC, significantly outperforming prior weakly-supervised methods (54.2 mDSC).

Qualitatively, RLSP demonstrates a superior ability to capture the full extent of objects. While traditional Class Activation Map (CAM) methods often highlight only the most distinct features (such as the head of an animal), RLSP successfully delineates the entire body by treating every unclaimed object patch as a potential reward for the competing agent.

Discussion and Limitations

RLSP offers a significant shift from traditional WSS by replacing static thresholding with a dynamic, learned search process. However, the method's performance is tied to two main factors:

  1. Patch Size: The resolution of the segmentation is limited by the grid size. Very small patches may lack the context needed for the detector to identify the object, while very large patches result in coarse boundaries.
  2. Detector Quality: The agents are only as effective as the "referee." Biases in the initial image-level classifier can propagate through the RL training process.

Future work aims to address these by exploring end-to-end training of the detector and agents simultaneously, as well as extending the competitive discovery framework to 3D localization and anomaly detection tasks.

Summary updated

Abstract

Weakly-supervised semantic segmentation (WSSS) methods, reliant on image-level labels indicating object presence, lack explicit correspondence between labels and regions of interest (ROIs), posing a significant challenge. Despite this, WSSS methods have attracted attention due to their much lower annotation costs compared to fully-supervised segmentation. Leveraging reinforcement learning (RL) self-play, we propose a novel WSSS method that gamifies image segmentation of a ROI. We formulate segmentation as a competition between two agents that compete to select ROI-containing patches until exhaustion of all such patches. The score at each time-step, used to compute the reward for agent training, represents likelihood of object presence within the selection, determined by an object presence detector pre-trained using only image-level binary classification labels of object presence. Additionally, we propose a game termination condition that can be called by either side upon exhaustion of all ROI-containing patches, followed by the selection of a final patch from each. Upon termination, the agent is incentivised if ROI-containing patches are exhausted or disincentivised if a ROI-containing patch is found by the competitor. This competitive setup ensures minimisation of over- or under-segmentation, a common problem with WSSS methods. Extensive experimentation across four datasets demonstrates significant performance improvements over recent state-of-the-art methods.
BibTeX
@article{saeed2025competing,
  author = {Shaheer U. Saeed and Shiqi Huang and João Ramalhinho and Iani J. M. B. Gayo and Nina Montaña Brown and Ester Bonmati and Stephen P. Pereira and Brian R. Davidson and Dean C. Barratt and Matthew J. Clarkson and Yipeng Hu},
  title = {{Competing for Pixels: A Self-Play Algorithm for Weakly-Supervised Semantic Segmentation}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2025},
  doi = {10.1109/tpami.2024.3474094},
}

Loading related content...

Figure