Interpretable Actions: Controlling Experts with Understandable Commands

Shumeet Baluja, David Marwood, Michele CovellAAAI 2021 · DOI

Introduction

Deep neural networks are often criticized as "black boxes," where the internal logic leading to a specific output remains opaque. This lack of interpretability is particularly problematic in creative or technical domains where the output should ideally be a composition of known, discrete steps—such as a series of brushstrokes in a painting or a sequence of geometric primitives in a 3D model.

Researchers from Google have introduced a novel two-stage framework that addresses this by shifting the focus from internal weights to interpretable actions. Instead of generating a monolithic grid of pixels, the system learns to manipulate external tools through understandable commands, ensuring that every result is fully decomposable into human-readable instructions.

The Two-Stage Behaviorist Framework

The core of the proposed method is a "behaviorist" approach that bridges the gap between differentiable neural networks and non-differentiable real-world tools (like graphics engines).

Stage 1: Training Differentiable Proxies

Many professional tools cannot be used directly in a standard training loop because they are non-differentiable. To circumvent this, the authors first train small "generator modules"—neural network twins—to mimic the behavior of these external tools. These proxies are differentiable, allowing gradients to flow through them during the next stage of training.

Stage 2: Training the Controller

A larger controller network is then trained to output parameters (commands) for these generators. During this stage, the generators are frozen. The controller learns to select the optimal parameters so that the combined output of the generators matches a specific target goal.

Overview of the reconstruction process across different image types, showing the progression from initial actions to final output.

Iterative Refinement and Action Extraction

To achieve high fidelity, the system employs an iterative residual correction process. The controller evaluates the error (the residual) between its current attempt and the target, then generates additional actions to correct those specific mistakes.

Once the controller has finished, the resulting parameters are extracted. Because the generators were trained to mimic real-world tools, these parameters can be plugged directly into the original, non-differentiable external tools (e.g., a vector rendering engine) to recreate the result with perfect precision. This ensures that the AI's "logic" is preserved in a format humans can inspect and edit.

Experimental Validation

The researchers tested this framework across three distinct domains to prove its versatility:

Artistic Style Transfer

In the artistic domain, the system was tasked with recreating photographs using crayon-like strokes. Rather than simply blurring or filtering pixels, the network learned to place thousands of individual Bezier curves. This process effectively captures the "logic" of drawing, identifying dominant features and representing them through discrete actions.

Detailed breakdown of a portrait reconstruction using individual crayon-like strokes.

Inverse Rendering and Function Approximation

For inverse rendering, the system deconstructed complex images, such as American Gothic, into 500 basic geometric shapes like ellipses and rectangles. On the ImageNet test set, the extracted commands showed high accuracy, particularly when refined with Least-Squares Error (LSE) re-weighting. In function approximation tasks, the iterative residual approach reduced error by 29% compared to standard single-pass baselines.

Conclusion and Future Directions

By forcing neural networks to operate through a bottleneck of interpretable actions, this framework provides a clear window into the AI's decision-making process. While the current system requires users to specify the number of actions, future iterations could incorporate a "parsimony factor" to automatically determine the minimum number of steps needed for a task. This research marks a significant step toward AI systems that are not only powerful but also fundamentally understandable to the humans who use them.

Summary updated

BibTeX
@inproceedings{baluja2021interpretable,
  author = {Shumeet Baluja and David Marwood and Michele Covell},
  title = {{Interpretable Actions: Controlling Experts with Understandable Commands}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i6.16624},
}
Figure