Interpreting Models by Allowing to Ask

Sungmin Kang, David Keetae Park, Jaehyuk Chang, Jaegul Choo

Abstract

Deep learning models are notoriously opaque, but what if we could understand them by letting them ask questions? This paper introduces "asking networks," a framework where a model iteratively requests specific hints about parts of a task it finds confusing. Applied to image colorization, the model naturally learns to ask for the colors of semantically meaningful objects, revealing its inner uncertainties and outperforming traditional interactive colorization methods.

Introduction

Deep neural networks are often treated as black boxes. While post-hoc visualization tools like saliency maps can highlight which input pixels influenced a decision, they fail to communicate what information the model feels it is missing. In multimodal "one-to-many" tasks—such as image colorization, where a single grayscale image could have many valid color palettes—it is difficult to discern why a model chose a specific output. This research addresses the interpretability gap by shifting the paradigm: instead of just observing the model passively, the authors allow the model to communicate its uncertainty by asking questions during the prediction process.

Methods: The Asking Network Framework

The authors propose a general framework called asking networks. In this setup, a model does not simply output a final prediction; instead, it sequentially generates a "question" and receives an "answer" before refining its guess.

For the task of image colorization, the model outputs a spatial heatmap (values from 0 to 1) indicating a specific region it wants to know the color of. An external function then provides the average ground-truth color of that region. Using a U-Net architecture, the model updates its intermediate state and asks subsequent questions based on the new information.

The iterative asking process using a U-Net architecture with residual connections.

To ensure the model asks interpretable and useful questions, two training techniques are employed:

  1. Smoothing Loss: Encourages the model to generate spatially coherent heatmaps rather than scattered pixels.
  2. Noise Injection: Prevents the model from asking for the average color of multiple distinct objects by injecting random noise into the answers, forcing it to isolate single, distinct objects for reliable hints.

Key Insights and Results

The model exhibits "greedy curiosity," consistently targeting high-error regions first. Interestingly, the act of formulating useful questions leads to emergent segmentation; the model learns to identify semantically coherent objects (like a person's shirt) because they provide the most mathematically useful hints for minimizing loss.

Sequential colorization process showing the model's questions (heatmaps) and the resulting color updates over time.

Quantitatively, the asking network significantly outperforms existing interactive colorization baselines. With just one hint, the proposed model achieves a PSNR of 28.16, compared to 25.37 for the next best baseline. Furthermore, on the VOC 2012 dataset, the regions the model asks about overlap with a single ground-truth semantic class 86.7% of the time, proving that the questions are semantically meaningful to humans.

Conclusion

While training asking networks is computationally intensive, the framework provides a novel path toward interpretable AI. By allowing models to "ask," we gain insight into their internal decision-making and provide a mechanism for human users to steer outputs. Future work may extend this paradigm to other domains, such as image inpainting or natural language processing, where resolving ambiguity is critical.

Summary updated

Abstract

Questions convey information about the questioner, namely what one does not know. In this paper, we propose a novel approach to allow a learning agent to ask what it considers as tricky to predict, in the course of producing a final output. By analyzing when and what it asks, we can make our model more transparent and interpretable. We first develop this idea to propose a general framework of deep neural networks that can ask questions, which we call asking networks. A specific architecture and training process for an asking network is proposed for the task of colorization, which is an exemplar one-to-many task and thus a task where asking questions is helpful in performing the task accurately. Our results show that the model learns to generate meaningful questions, asks difficult questions first, and utilizes the provided hint more efficiently than baseline models. We conclude that the proposed asking framework makes the learning agent reveal its weaknesses, which poses a promising new direction in developing interpretable and interactive models.
BibTeX
@misc{kang2018interpreting,
  author = {Sungmin Kang and David Keetae Park and Jaehyuk Chang and Jaegul Choo},
  title = {{Interpreting Models by Allowing to Ask}},
  year = {2018},
  eprint = {1811.05106},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.05106},
}
Figure