Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance

Ioannis Kalogeropoulos, Giorgos Bouritsas, Yannis PanagakisSubmission to ICLR 2026 · Rejected

TL;DR

Instead of painfully retraining or fine-tuning models when new regulatory or performance requirements arise, this paper introduces a data-driven approach: using a graph metanetwork to edit neural networks in a single forward pass. By framing regulatory compliance as a multi-objective problem, the metanetwork acts as a fast, post-hoc "editor" that satisfies constraints (such as data minimization, fairness, or sparsity) while preserving the model’s original behavior in roughly 30 milliseconds.

What Problem Does This Paper Address?

Regulatory standards (like the EU AI Act or GDPR), fairness requirements, and hardware constraints often emerge after a machine learning model has already been trained and deployed. When an auditor or developer must ensure compliance, they typically face three unpleasant choices:

  1. Retraining from scratch: Prohibitively expensive and slow.
  2. Fine-tuning: Time-consuming and often impossible if the auditor lacks access to proprietary training data.
  3. Traditional post-processing: Fast, but usually degrades model utility significantly.

The authors ask: Can we learn an operator that directly maps a non-compliant neural network's weights to compliant weights in a single step, without sacrificing performance?

Methods

The authors cast requirement compliance as a multi-objective optimization problem with two competing goals:

  1. Preservation Objective: Minimize functional drift from the original model's outputs using expected Jensen–Shannon Divergence ($\text{JSD}$).
  2. Requirement Objective: Enforce compliance (e.g., penalizing used input features, equalized odds differences, or unpruned weights).

Rather than solving this optimization for each model individually, they train a Graph Metanetwork (GMN)—a meta-model that takes a neural network’s computational graph and weights as input and directly predicts modified weights (or masks) in one forward pass:

  • Symmetry-Aware Architecture: Neural networks have permutation symmetries (e.g., permuting hidden neurons doesn't change functionality). The authors build upon bidirectional graph neural networks with specialized positional encodings to respect parameter symmetries and architecture differences.
  • Differentiable Relaxations: Discrete operations (like masking input features or pruning connections) are relaxed during training using temperature-scaled softmax and straight-through estimators, enabling end-to-end gradient-based meta-learning.
  • Residual Parameter Updates: For pruning and feature removal, the metanetwork predicts both the selection mask and residual adjustments ($\Delta \theta$) for the remaining parameters to compensate for removed components.

Datasets

The authors evaluate their approach on populations of multi-layer perceptrons (MLPs) trained on two standard UCI tabular benchmarks:

  • Adult (Census Income): Used for data minimization, bias mitigation (gender fairness), and weight pruning.
  • Bank Marketing: Used for data minimization and weight pruning.

Model Populations ($D_m$): The authors generated a benchmark of 12,000 model checkpoints per task, sampled across varied depths (1–4 hidden layers), widths (32–64), activations, normalizations, and regularizers. Crucially, the metanetwork is trained on a small probe subset of data points ($D_d$) rather than the full original training set.

Key Insights and Ideas

  • Editing in a Single Step ($\approx 0.03\text{s}$): By learning the mapping from original weights to Pareto-optimal weights, model editing shifts from an iterative optimization routine to a single inference forward pass, running in ~0.03 seconds per model (versus 32+ seconds for distillation baselines).
  • Joint Masking and Compensation: Traditional pruning or feature dropping hurts accuracy unless followed by retraining. The GMN dynamically adjusts the surviving weights in tandem with masking, recovering functionality instantly.
  • Composability of Metanetworks: Metanetworks trained independently on different requirements can be chained together sequentially (e.g., $\mathcal{F}_{\text{prune}} \circ \mathcal{F}_{\text{minimize}}$). Because each network preserves the underlying model distribution, stacking them yields a joint 3D Pareto front that still outperforms composite baselines.
  • Data Privacy for Auditors: Third-party auditors do not need the original training data. The metanetwork learns to edit models using only a small validation probe set.

Key Results

  • Pareto Dominance Across Tasks: In all three settings (Data Minimization, Fairness/Equalized Odds, and Weight Pruning), the proposed GMN consistently dominated existing baselines along the Pareto front (JSD drift vs. compliance metric).
  • Speedup: On the Adult dataset, editing via GMN took 0.03s, compared to 32.35s for feature selection with retraining, 4.36s for RejectOption fair classification, and 0.41s for SNIP pruning.
  • Sample Efficiency: An ablation over training population size showed that using only 25% of the model population was sufficient for the metanetwork to reach near-optimal Pareto performance.

Limitations and Future Work

  • Task Specialization: The current formulation trains an editor for a fixed task/data distribution ($p_d$). A truly universal editor capable of multi-task editing across arbitrary modalities and domains remains an open challenge.
  • Architecture Constraints: Experiments were limited to MLPs on tabular data; scaling to large convolutional backbones, Transformers, and LLMs is left for future work.
  • Need for Human Oversight: The authors note that while the method produces Pareto fronts to navigate trade-offs, human judgment is still essential to decide acceptable balance points in high-stakes sociotechnical settings.

Summary updated

Abstract

As machine learning models are increasingly deployed in high-stakes settings, e.g. as decision support systems in various societal sectors or in critical infrastructure, designers and auditors are facing the need to ensure that models satisfy a wider variety of requirements (e.g. compliance with regulations, fairness, computational constraints) beyond performance. Although most of them are the subject of ongoing studies, typical approaches face critical challenges: post-processing methods tend to compromise performance, which is often counteracted by fine-tuning or, worse, training from scratch, an often time-consuming or even unavailable strategy. This raises the following question: "Can we efficiently edit models to satisfy requirements, without sacrificing their utility?" In this work, we approach this with a unifying framework, in a data-driven manner, i.e. we learn to edit neural networks (NNs), where the editor is an NN itself - a graph metanetwork - and editing amounts to a single inference step. In particular, the metanetwork is trained on NN populations to minimise an objective consisting of two terms: the requirement to be enforced and the preservation of the NN's utility. We experiment with diverse tasks (the data minimisation principle, bias mitigation and weight pruning) improving the trade-offs between performance, requirement satisfaction and time efficiency compared to popular post-processing or re-training alternatives.
BibTeX
@misc{kalogeropoulos2025metanetworks,
  author = {Ioannis Kalogeropoulos and Giorgos Bouritsas and Yannis Panagakis},
  title = {{Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance}},
  year = {2025},
  eprint = {2512.15469},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.15469},
}
Figure