Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference

Jorge García-Carrasco, Alejandro Maté, Juan TrujilloAAAI 2025 · DOI

TL;DR

This paper introduces a novel method to automatically extract minimal, task-specific "circuits" from large language models (LLMs). This allows for significantly smaller and faster models that retain performance on the target task, making LLMs more accessible for resource-constrained environments and easier to understand.

What Problem Does This Paper Address?

Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide array of tasks. However, their sheer size (billions of parameters) makes them computationally expensive to deploy, requiring substantial memory and processing power. This limits their use in many practical applications, especially those with resource constraints. Furthermore, when an LLM is used for a specific task, its vast general knowledge is often unnecessary, leading to inefficient computation. While Mechanistic Interpretability (MI) research has shown that specific tasks are handled by localized "circuits" within LLMs, existing MI techniques primarily focus on understanding these circuits rather than extracting them for standalone use. This paper aims to bridge this gap by proposing a method to automatically extract these task-specific circuits, creating smaller, faster, and more interpretable models.

Methods

The core of this paper is a novel MI-based pruning approach. The authors leverage the idea that specific tasks are performed by identifiable circuits within an LLM. Their method works by iteratively "patching" (temporarily altering) components of the LLM and observing the impact on the model's output for a specific task.

Here's a breakdown of their approach:

  • Initialization: The process starts with the full LLM and a dataset designed to elicit the target task's behavior.
  • Iterative Pruning: The algorithm traverses the LLM's components (attention heads and optionally MLPs) from the last layer backward.
  • Patching and Evaluation: For each component, its activations are temporarily ablated (either set to zero or replaced with their mean value across a reference distribution). The impact of this ablation is measured by calculating the Kullback-Leibler (KL) divergence between the original model's predictions and the ablated model's predictions on a validation dataset.
  • Pruning Decision: If the difference in KL divergence between the ablated and original model is below a certain threshold ($\alpha$), it indicates that the component is not crucial for the specific task and can be permanently pruned.
  • Standalone Extraction: Unlike traditional MI methods that only identify circuits, this approach removes the non-essential components, resulting in a smaller submodel. This submodel can then be used independently without the need for further training or fine-tuning.

The key novelty lies in using MI principles (patching and KL divergence) not just for understanding, but for automated, task-specific pruning to create a functional, smaller model.

Datasets

The paper uses datasets specifically curated to elicit the behavior of three distinct tasks. These datasets are split into patching and validation sets. The specific datasets are not detailed in terms of their creation process beyond their purpose, but they are used to evaluate the performance of the extracted circuits.

Key Insights and Ideas

  • MI for Practical Compression: The most significant insight is demonstrating that Mechanistic Interpretability can be directly applied to create practical model compression techniques, moving beyond theoretical understanding.
  • Task-Specific Extraction is Key: The paper highlights that for many applications, a full LLM is overkill. Extracting only the necessary "circuit" for a specific task offers substantial benefits in efficiency and interpretability.
  • Mean Ablation vs. Zero Ablation: Mean ablation (replacing activations with their average) proves to be a more robust and less aggressive pruning strategy than zero ablation. Zero ablation can be too disruptive, leading to larger pruned models to maintain accuracy.
  • Prioritizing Attention Heads: The pruning process naturally prioritizes pruning attention heads before MLPs, suggesting that attention mechanisms are often more task-specific and can be removed more readily.
  • Small Data, Big Impact: The proposed method can effectively extract task-specific circuits using relatively small datasets, which are insufficient for traditional knowledge distillation but ample for this MI-based approach.

Key Results

  • Significant Size Reduction: The pruned models achieved substantial parameter reductions, up to 82.77%, making them considerably smaller and faster.
  • Performance Preservation: The extracted circuits were able to perform the target tasks with accuracy comparable to, and sometimes even exceeding, the original LLM. For example, for the "Greater-than" task, a model reduced by 82.77% maintained 99.84% accuracy.
  • Interpretability Maintained: The resulting pruned models are more interpretable because they focus on the specific components responsible for the task, aligning with manually identified circuits from previous MI research.
  • Comparison to Distillation: The proposed method outperformed knowledge distillation on tasks requiring small datasets, as distillation typically needs more data to train a student model effectively.

Limitations and Future Work

  • Model and Task Specificity: The current evaluation is limited to GPT-2 Small and three fixed-sequence length, single-token prediction tasks.
  • Future Work: The authors suggest extending the evaluation to more complex tasks, including multi-token prediction, generative tasks, and sentiment analysis. They also plan to test their approach on larger, production-grade LLMs. Additionally, exploring different ablation schemes and pruning strategies could further enhance the method.

Summary updated

BibTeX
@inproceedings{garciacarrasco2025extracting,
  author = {Jorge García-Carrasco and Alejandro Maté and Juan Trujillo},
  title = {{Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33843},
}
Figure