Abstract
This paper addresses the challenge of decomposing large-scale spatial tasks, such as robotic painting or surveying, into manageable segments. We frame this problem as a hierarchical image parsing task and introduce DRAG (DeteRministically AGgrevate), a novel deterministic actor-critic imitation learning algorithm. DRAG enables a neural parser to learn from an expert decision-tree oracle using raw image data. Our approach significantly outperforms standard reinforcement learning and imitation learning baselines, achieving 86.86% test accuracy on a custom object parsing dataset.
Introduction
Robotic agents often face spatial tasks that exceed the physical footprint of their tools. For instance, a painting robot must decompose a large object into smaller, executable segments to ensure full coverage while minimizing overlap. Finding the optimal recursive partitioning is complex. While classical decision tree algorithms like Information Gain Maximization (IGM) can partition space effectively, they require ground-truth labels at inference time—data unavailable to a robot in a novel environment. We propose training a neural network to "parse" raw images into hierarchical segments, imitating the IGM oracle's behavior without requiring ground truth during deployment.
The DRAG Framework
We formulate shape parsing as a Markov Decision Process (MDP). At each step, the parser observes an image segment and selects a discrete splitting rule (horizontal or vertical) and a continuous split location. A key insight of this work is that hierarchical parse-trees used in grammar-based image parsing are structurally identical to the decision trees built by classical IGM algorithms.
To handle the continuous action space, we introduce DRAG (DeteRministically AGgrevate). DRAG is a deterministic actor-critic variant of the AggreVaTeD algorithm. It utilizes a differentiable critic network to approximate the expert oracle's cost-to-go, allowing for deterministic policy gradient updates. Unlike standard Deterministic Policy Gradients (DPG) where the critic evaluates the learner's policy, DRAG's critic evaluates the expert oracle's cost-to-go. This removes the need for approximation, yielding a cleaner, truer gradient and leading to more regular, boundary-aligned parses.
Experimental Evaluation
We evaluated DRAG on a custom dataset of 362 RGB object images annotated with per-pixel binary labels. The models were tested using 3-fold cross-validation, comparing DRAG against standard Reinforcement Learning (RL) and Imitation Learning (IL) baselines.
Key Results
- Quantitative Performance: DRAG achieved a test accuracy of 86.86%, significantly higher than pure RL baselines like DDPG (~48.78%) and standard imitation learning methods like DAgger (84.03%).
- RL vs. IL: Pure RL failed due to the recursive nature of parsing; a single early error propagates through the tree, making exploration difficult. Imitation learning provides the necessary guidance for these critical early splits.
- Qualitative Superiority: As seen in the qualitative results, DRAG produces cleaner splits that align with prominent image gradients. Notably, DRAG demonstrated the ability to generalize beyond the expert oracle, correctly labeling fine details that the depth-limited IGM oracle missed on complex shapes like car doors.
Conclusion
DRAG provides a robust framework for learning hierarchical spatial decompositions from raw visual input. By combining the strengths of deterministic policy gradients with expert-guided imitation learning, we achieve high-fidelity parsing that generalizes to novel objects. Future work will explore extending this approach to 3D spatial tasks and non-rectilinear splitting grammars.
Summary updated