Active Object Localization with Deep Reinforcement Learning

Juan C. Caicedo, Svetlana Lazebnik

Abstract

We present an active detection model for localizing objects in scenes. The model is class-specific and allows an agent to focus attention on candidate regions for identifying the correct location of a target object. This agent learns to deform a bounding box using simple transformation actions, with the goal of determining the most specific location of target objects following top-down reasoning. The proposed localization agent is trained using deep reinforcement learning, and evaluated on the Pascal VOC 2007 dataset. We show that agents guided by the proposed model are able to localize a single instance of an object after analyzing only between 11 and 25 regions in an image, and obtain the best detection results among systems that do not use object proposals for object localization.

Introduction

Traditional object detection pipelines typically rely on "bottom-up" strategies, such as sliding windows or object proposal algorithms (e.g., Selective Search). These methods generate thousands of candidate regions that must be processed by a heavy classifier, which is computationally expensive and differs significantly from human visual perception. Humans utilize active attention and top-down reasoning to quickly focus on relevant areas of a scene.

This paper proposes a paradigm shift: treating object localization as a sequential decision-making process. By framing localization as an agent navigating an environment, the model learns a policy to "zoom in" on a target object through a series of discrete transformations, significantly reducing the number of regions evaluated per image.

Methodology

The localization task is modeled as a Markov Decision Process (MDP) solved via Deep Q-Learning (DQN).

The Agent and Environment

The environment consists of the input image, while the agent controls the coordinates of a bounding box. The agent's goal is to transform this box until it tightly frames the target object.

Sequential actions of the agent localizing a cow instance.

State Representation and Action Space

The state at each step $t$ is defined by visual features extracted via a pre-trained CNN (Zeiler-Fergus architecture) and a history of the last 10 actions to prevent cyclic behavior. The agent has 9 possible actions: four translation moves (Up, Down, Left, Right), two scale changes (Bigger, Smaller), two aspect ratio adjustments (Fatter, Taller), and a Trigger action to finalize the detection.

Rewards and Multi-Instance Detection

The reward function encourages actions that increase the Intersection-over-Union (IoU) with the ground truth. A positive reward (+1) is given if IoU increases, and a negative reward (-1) otherwise. The "Trigger" action provides a large positive reward if IoU > 0.6. To detect multiple instances, an "Inhibition of Return" (IoR) mechanism marks triggered regions as "seen," allowing the agent to continue searching for other objects of the same class without getting stuck on the first discovery.

Results and Discussion

The model was evaluated on the Pascal VOC 2007 and 2012 datasets.

Efficiency and Recall

A key advantage of this approach is its sparsity. While proposal-based methods like R-CNN evaluate ~2,000 regions, this agent typically localizes an object in just 11 to 25 steps. As shown in the recall analysis, the proposed method achieves significantly higher recall than traditional baselines when the number of allowed proposals is very low (e.g., < 100).

Recall at IoU threshold 0.5 vs. number of proposals. The proposed method outperforms baselines in the low-proposal regime.

Accuracy and Performance

On Pascal VOC 2007, the system achieved 46.1 mAP, which was the state-of-the-art for methods not relying on external object proposals at the time of publication. On a K-40 GPU, the agent processes each region in approximately 7.74ms, making the total search time highly competitive. The agent also demonstrated class-specific search strategies, learning contextual cues relevant to specific categories, such as finding cows on grass or faces at the top of a person's bounding box.

Conclusion

Active object localization using Deep Reinforcement Learning provides a more efficient, biologically inspired alternative to exhaustive search methods. By transforming detection into a control problem, the agent achieves high recall with minimal region evaluations. Future work includes end-to-end training of the CNN features and improving performance on small objects in cluttered scenes.

Summary updated

Abstract

We present an active detection model for localizing objects in scenes. The model is class-specific and allows an agent to focus attention on candidate regions for identifying the correct location of a target object. This agent learns to deform a bounding box using simple transformation actions, with the goal of determining the most specific location of target objects following top-down reasoning. The proposed localization agent is trained using deep reinforcement learning, and evaluated on the Pascal VOC 2007 dataset. We show that agents guided by the proposed model are able to localize a single instance of an object after analyzing only between 11 and 25 regions in an image, and obtain the best detection results among systems that do not use object proposals for object localization.
BibTeX
@inproceedings{caicedo2015active,
  author = {Juan C. Caicedo and Svetlana Lazebnik},
  title = {{Active Object Localization with Deep Reinforcement Learning}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2015},
  url = {https://openreview.net/forum?id=H1ZicZfdbr},
}
Figure