Abstract
Scene Graph Generation (SGG) is often hindered by standard training objectives that fail to account for the structural importance of specific nodes or the non-differentiable nature of global metrics. We present Counterfactual critic Multi-Agent Training (CMAT), a framework that treats objects as agents in a cooperative game. By utilizing a counterfactual critic to solve the credit assignment problem, CMAT optimizes for graph-level coherence, achieving state-of-the-art results on the Visual Genome benchmark.
Introduction: The Challenge of Graph Coherence
Scene Graph Generation (SGG) involves transforming an image into a structured representation where nodes represent objects and edges represent their relationships. While object detection has matured, accurately capturing relationships remains difficult.
Current models suffer from two primary issues:
- Lack of Graph Coherence: Standard Cross-Entropy (XE) loss treats all errors equally. However, in a graph, misclassifying a "hub" node (e.g., a person interacting with multiple objects) is far more detrimental to the overall scene understanding than misclassifying an isolated leaf node.
- Local-Sensitivity Gap: Global metrics like Recall@K are non-differentiable. Reinforcement Learning (RL) can optimize these, but it faces the "credit assignment problem"—determining which specific object prediction contributed to a high global reward.
Methodology: CMAT and the Counterfactual Critic
CMAT addresses these challenges by framing SGG as a multi-agent reinforcement learning problem.
Objects as Agents
Each detected object is an agent that communicates with others via a module using LSTMs and attention mechanisms to share visual context. This ensures that predictions are contextually informed.
Graph-Level Rewards
Instead of local losses, CMAT maximizes a global reward (e.g., Recall@K). This forces the model to prioritize the overall quality and coherence of the generated graph.
The Counterfactual Critic
The core innovation is the counterfactual critic. To assign credit accurately, the model calculates a "counterfactual baseline": what the reward would have been if a specific object had a different label while others remained unchanged. The difference between the actual reward and this baseline isolates the specific contribution (advantage) of that object's prediction.
Key Insights
- Cooperative Learning: Framing SGG as a multi-agent task acknowledges the codependency of labels. For instance, a "man" is statistically more likely to be "riding" a "bike" than a "car."
- Structural Importance: By optimizing for the whole graph, CMAT learns that "hub" nodes matter more. Errors in central nodes lead to significant drops in global reward, incentivizing the model to be more accurate where it counts most.
- Contextual Disentanglement: The communication module focuses on context gathering, while the policy gradient handles the objective, preventing early saturation issues common in message-passing networks.
Results and Evaluation
Evaluated on the Visual Genome (VG) dataset, CMAT demonstrated superior performance:
- Scene Graph Classification (SGCls): Achieved an absolute improvement of 3.4% to 4.3% in Mean Recall over previous joint inference models.
- Ablation Studies: The counterfactual baseline significantly outperformed standard RL baselines like "Moving Average," proving the necessity of precise credit assignment.
- Qualitative Analysis: CMAT is notably better at identifying central objects in complex scenes compared to models like MOTIFS, which often struggle with hub consistency.
Conclusion and Future Work
While CMAT offers a powerful framework for structural learning, it faces computational challenges. Calculating the counterfactual baseline is expensive, requiring a "top-k" approximation for feasibility. Additionally, performance remains tied to the quality of the underlying object detector (e.g., Faster R-CNN). Future work may extend this counterfactual multi-agent approach to other structured tasks like Visual Question Answering (VQA) and Image Captioning.
Summary updated