Programmatic and Symbolic Distillation for Interpretable Reinforcement Learning

This direction is split into other directions: policy interpretability. Its last version stays readable and citable here.

Version 1 of 1109 papers110 conceptsfrozen 2026-10-04

Deep Reinforcement Learning (DRL) has achieved remarkable success in complex domains, yet its reliance on high-dimensional neural networks creates a "black box" problem. In safety-critical fields like healthcare, finance, and autonomous robotics, a policy that works is not enough; human operators must understand why it works to ensure safety, accountability, and trust.

To bridge this gap, a growing body of research focuses on distilling opaque neural policies into structured, human-readable representations—such as computer programs, logical rules, or symbolic trees.

The Transparency Gap in Deep RL

Standard DRL agents store their decision-making logic in millions of numerical weights. As noted by Claire Glanois and Paul Weng, this lack of transparency is a primary barrier to real-world adoption in high-stakes environments. If a self-driving car makes a dangerous maneuver, an engineer cannot simply "read" the neural network to diagnose the failure.

Researchers like Hector Kohler argue that for AI to be truly reliable, its policies must be auditable and editable. This has led to the development of methods that transform "System 1" intuition (neural networks) into "System 2" logic (symbolic reasoning), a direction surveyed extensively by K. Acharya and Waleed Raza.

Distilling Policies into Programs

One powerful approach is Programmatic Reinforcement Learning, where the goal is to represent a policy as a snippet of source code. Programs are inherently modular, interpretable, and can be formally verified.

  • Hierarchical Composition: Guan-Ting Liu developed the HPRL framework, which trains a meta-policy to compose sequences of human-readable sub-programs. This allows the AI to solve complex tasks by chaining together simple, understandable logic.
  • Imitation and Generalization: Manuel Eberhardinger and Johannes Maucher demonstrated that distilling multi-agent neural policies into logical programs not only improves interpretability but also allows the agents to generalize much better to larger, unseen environments than their neural counterparts.
  • Physics-Informed Synthesis: To make the search for programs more efficient, Arjun Guha and Joydeep Biswas introduced LDIPS, which uses dimensional analysis (the laws of physics) to prune the space of possible programs, ensuring the synthesized policies are physically grounded.

Symbolic Logic and Decision Trees

Beyond code, researchers use symbolic logic and decision trees to provide a "window" into the agent's mind.

Zhengyao Jiang and Shan Luo proposed Neural Logic Reinforcement Learning (NLRL), which represents policies using first-order logic. By using a differentiable logic engine, the agent learns rules like "if the goal is north and the path is clear, move north," which are immediately understandable to humans. Similarly, Alberto Castellini uses Inductive Logic Programming to extract "rules of thumb" from execution data, providing heuristics that match expert performance.

In the networking domain, S P Sharan developed SymbolicPCC, which distills complex TCP congestion control agents into tiny symbolic trees. These trees are not only interpretable but also 23x faster to execute in an OS kernel than neural networks.

Neuro-Symbolic Hybrids and Visualization

Rather than replacing neural networks entirely, some researchers combine them with symbolic components. Daiki Kimura and colleagues introduced Logical Optimal Actions (LOA), which uses Logical Neural Networks (LNNs) to provide visual explanations of reasoning in text-based games.

For existing "black-box" models, post-hoc interpretation tools are used to "gray the box." Tom Zahavy and Shie Mannor pioneered the use of t-SNE visualizations and saliency maps to reveal that Deep Q-Networks (DQNs) actually learn sophisticated hierarchical abstractions, even if they aren't explicitly programmed to do so.

Applications in High-Stakes Domains

The push for interpretability is driven by specific industry needs:

  • Finance: Shuo Sun and Bo An emphasize that explainable trade actions are a prerequisite for institutional trust in RL-based quantitative trading.
  • Power Systems: Peng Li highlights that grid operators cannot deploy autonomous agents without formal safety guarantees and interpretable logic.
  • Education: Adish Singla notes that pedagogical AI must provide rationales for instructional decisions to be accepted by teachers and students.

By moving from opaque weights to transparent structures, this research direction ensures that as reinforcement learning agents become more powerful, they also become more accountable to the humans they serve.

Papers in this version

Showing 60 of 109 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_programmatic_and_symbolic_distillation_f_2b5d095a,
  title = {Programmatic and Symbolic Distillation for Interpretable Reinforcement Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/programmatic-and-symbolic-distillation-for-interpretable-reinforcement-learning/rdr_2b5d095abc7b49d8a712a3ac8e372dfd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_programmatic_and_symbolic_distillation_f_2b5d095a_v1,
  title = {Programmatic and Symbolic Distillation for Interpretable Reinforcement Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/programmatic-and-symbolic-distillation-for-interpretable-reinforcement-learning/rdr_2b5d095abc7b49d8a712a3ac8e372dfd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}