MQA: Answering the Question via Robotic Manipulation

Yuhong Deng, Di Guo, Xiaofeng Guo, Naifu Zhang, Huaping Liu, Fuchun Sun

TL;DR

Manipulation Question Answering (MQA) is a new task where robots don't just look at a scene to answer questions—they physically interact with it. By using a Deep Q-Network (DQN) to perform targeted pushes, a robot can uncover occluded objects in cluttered environments to accurately answer natural language queries like "How many cans are in the bin?"

What Problem Does This Paper Address?

Traditional Visual Question Answering (VQA) is a passive process: an agent looks at a static image and provides an answer. While Embodied Question Answering (EQA) moved this into 3D space by allowing agents to navigate, it still lacked the ability to physically alter the environment. Interactive Question Answering (IQA) introduced simple actions like opening a refrigerator, but real-world environments are often cluttered and unstructured.

In a cluttered bin, a target object might be hidden beneath others. A passive observer would fail to see it, and a simple navigator wouldn't be able to reach it. The authors propose Manipulation Question Answering (MQA) to bridge this gap, requiring the robot to generate a sequence of manipulation actions (like pushing) to change the environment specifically to find the information needed for an answer.

The MQA task: The robot pushes a box to reveal a hidden third can, allowing it to correctly answer "Three."

Methods

The researchers developed a framework consisting of two primary components:

  1. Manipulation Module: This module uses a Deep Q-Network (DQN) to decide how the robot should interact with the scene.
  2. State Space: A combination of RGBD images (processed via ResNet101) and the question (encoded via a 2-layer LSTM).
  3. Action Space: The robot can choose from 8 different push directions at any location or a "stop" action once it believes it has enough information.
  4. Reward Design: This is the "secret sauce." The reward function balances exploration (moving objects) with a task-specific reward based on reducing the overlap rate (occlusion) of objects. They tested three reward strategies: a global reward ($R_g$) for simplifying the whole scene, a local reward ($R_l$) focusing on the most occluded object, and a hybrid of both.
  5. QA Module: Once the manipulation module triggers a "stop," the QA module takes over. It uses the initial scene, the final scene, and the question to predict the answer. It employs an attention mechanism to fuse image features with the question encoding before passing them to a softmax classifier.

Datasets

Because MQA is a novel task, no existing datasets were suitable. The authors built the MQA Dataset using the V-REP robotic simulator:

  • Environment: A UR5 robotic arm equipped with a gripper and a Kinect camera overlooking a bin.
  • Objects: 60 different 3D models across 20 common categories (e.g., cans, books, gamepads, tools).
  • Scenes: 100 scene series categorized by difficulty (Easy, Medium, Hard) based on the number of objects (20, 35, or 50).
  • Questions: 2,000 questions per type, covering COUNTING, EXISTENCE, SPATIAL, and LOGIC queries.
Overview of the MQA dataset, including the simulation environment and object models.

Key Insights and Ideas

  • Active Perception via Manipulation: The core insight is that manipulation is a tool for perception. By pushing a "distractor" object, the robot actively reduces the entropy of the scene relative to the question.
  • Local vs. Global Focus: The experiments revealed that focusing on the most occluded object ($R_l$) is generally more effective than trying to simplify the entire scene ($R_g$). In complex, cluttered environments, targeted intervention is more efficient than broad changes.
  • Dynamic Reward Weighting: The authors used a time-varying weight ($\beta$) for the exploration reward. Early in training, the robot is rewarded simply for moving things ($R_e$). As training progresses, the reward shifts toward actions that specifically help answer the question ($R_q$).

Key Results

The MQA system was evaluated primarily on COUNTING questions, which are considered the most difficult as they require thorough exploration.

  • Baseline Comparison: The DQN-based models significantly outperformed a random manipulation baseline across all difficulty levels.
  • Reward Performance: The $DQN(R_l)$ model (local reward) achieved the highest accuracy in "Hard" scenes (52.89% vs. 27.35% for random).
  • Efficiency: The $DQN(R_g)$ model (global reward) performed best in "Easy" scenes, suggesting that when there are few objects, a general simplification of the scene is sufficient.

Limitations and Future Work

The authors identify a major hurdle: the lack of a recurrence mechanism. Currently, the manipulation module does not "remember" its previous actions. This can lead to "useless manipulations" where the robot repeats the same ineffective push because the state hasn't changed enough to trigger a different decision.

Future work aims to integrate RNNs or LSTMs into the manipulation policy to provide memory, allowing the robot to track which areas it has already explored and avoid repetitive actions. They also plan to expand the system to handle the other question types (Spatial, Logic, Existence) in the dataset.

For those interested in seeing the system in action, the authors have hosted an interactive demonstration at http://m-qa.acting-ai.com/.

Summary updated

Abstract

In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question. To solve this problem, a framework consisting of a QA module and a manipulation module is proposed. For the QA module, we adopt the method for the Visual Question Answering (VQA) task. For the manipulation module, a Deep Q Network (DQN) model is designed to generate manipulation actions for the robot to interact with the environment. We consider the situation where the robot continuously manipulating objects inside a bin until the answer to the question is found. Besides, a novel dataset that contains a variety of object models, scenarios and corresponding question-answer pairs is established in a simulation environment. Extensive experiments have been conducted to validate the effectiveness of the proposed framework.
BibTeX
@inproceedings{deng2021mqa,
  author = {Yuhong Deng and Di Guo and Xiaofeng Guo and Naifu Zhang and Huaping Liu and Fuchun Sun},
  title = {{MQA: Answering the Question via Robotic Manipulation}},
  booktitle = {Robotics: Science and Systems},
  year = {2021},
  doi = {10.15607/rss.2021.xvii.044},
}
Figure