Learning the Dynamics of Visual Relational Reasoning via Reinforced Path Routing

Chenchen Jing, Yunde Jia, Yuwei Wu, Chuanhao Li, Qi Wu

Abstract

Visual relational reasoning is traditionally approached through either holistic black-box models or brittle modular programs. This paper introduces a Reinforced Path Routing framework that treats reasoning as a sequential navigation task. By modeling the reasoning process as an agent "walking" through a visual graph, the approach achieves state-of-the-art performance on complex relational datasets while providing an inherently transparent and interpretable decision-making process.

The Challenge of Visual Reasoning

Complex visual queries, such as "What is the person to the left of the glasses holding?", require a sequence of logical steps. Humans naturally decompose these tasks: locating the glasses, identifying the adjacent person, and then inspecting the object in their hand.

Comparison of visual reasoning paradigms: (a) Inputs, (b) Modular, (c) Holistic, and (d) the proposed Path Routing.

Existing AI approaches often fall into two categories:

  1. Holistic Models: These process the entire image and query in a latent feature space. While powerful, they lack transparency, making it difficult to verify the reasoning chain.
  2. Modular Models: These assemble neural sub-programs. While more interpretable, they are often brittle and difficult to optimize for diverse, real-world scenarios.

The proposed framework addresses these limitations by explicitly representing reasoning dynamics as a path through a graph of visual objects.

Reinforced Path Routing Framework

The core of the proposed method is a three-stage process that transforms visual reasoning into a literal navigation problem.

1. Graph Construction

The input image is first converted into a visual graph $G$. Nodes represent objects detected via standard vision backbones (e.g., Faster R-CNN), while edges represent the spatial and semantic relationships between them.

2. The Reasoning Agent

The model acts as an agent navigating this graph. At each step, a Policy Network determines the next node to visit based on:

  • Language Attention: Identifying which parts of the query are relevant to the current step.
  • History Attention: A memory mechanism that tracks previous steps to ensure a coherent reasoning chain and prevent loops.
The sequential navigation process of the Path Routing agent, showing the step-by-step selection of nodes.

3. Training Strategy

To overcome the challenges of training reinforcement learning (RL) agents from scratch, a two-phase strategy is employed:

  • Supervised Warm-start: The agent begins with a "soft" navigation phase using a differentiable scaled softmax and a coverage loss to encourage exploration. A temperature parameter gradually "sharpens" the attention from soft distributions to hard node selections.
  • RL Refinement: Once the agent has a basic grasp of the task, it is further optimized using the Advantage Actor-Critic (A2C) algorithm, receiving rewards based on the accuracy of the final destination.

Key Insights and Results

The model was evaluated on CLEVR-Ref+, Ref-Reasoning, and GQA benchmarks.

  • Superior Accuracy: On the Ref-Reasoning dataset, the model achieved 53.02% accuracy, outperforming previous graph-based methods like SGMN and CMRIN. Its strength was particularly evident in complex queries involving four or more objects.
  • Compositional Strength: On GQA, the model reached 59.43% accuracy, demonstrating competitive performance on real-world compositional questions.
  • Interpretability: Unlike black-box models, the path routing agent provides a human-readable "walk" through the image, showing exactly which objects were considered at each step of the reasoning process.

Conclusion and Future Work

By treating reasoning as navigation, this work provides a bridge between the performance of holistic models and the transparency of modular ones. Future improvements could include a rollback mechanism to allow the agent to correct "wrong turns" and a dynamic step count that allows the agent to decide when it has sufficiently answered the query rather than relying on a fixed count derived from the sentence structure.

Summary updated

Abstract

Reasoning is a dynamic process. In cognitive theories, the dynamics of reasoning refers to reasoning states over time after successive state transitions. Modeling the cognitive dynamics is of utmost importance to simulate human reasoning capability. In this paper, we propose to learn the reasoning dynamics of visual relational reasoning by casting it as a path routing task. We present a reinforced path routing method that represents an input image via a structured visual graph and introduces a reinforcement learning based model to explore paths (sequences of nodes) over the graph based on an input sentence to infer reasoning results. By exploring such paths, the proposed method represents reasoning states clearly and characterizes state transitions explicitly to fully model the reasoning dynamics for accurate and transparent visual relational reasoning. Extensive experiments on referring expression comprehension and visual question answering demonstrate the effectiveness of our method.
BibTeX
@inproceedings{jing2022learning,
  author = {Chenchen Jing and Yunde Jia and Yuwei Wu and Chuanhao Li and Qi Wu},
  title = {{Learning the Dynamics of Visual Relational Reasoning via Reinforced Path Routing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i1.19997},
}
Figure