Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance

Ioannis Kalogeropoulos, Giorgos Bouritsas, Yannis Panagakis

TL;DR

This paper introduces a novel framework that uses "graph metanetworks"—neural networks trained to edit other neural networks—to instantly modify deployed models for regulatory compliance (such as fairness or privacy constraints). By treating model editing as a single forward pass, this approach eliminates the need for costly retraining while achieving a superior trade-off between requirement satisfaction and performance preservation.

What Problem Does This Paper Address?

As machine learning models are increasingly deployed in high-stakes environments, they must satisfy strict constraints beyond simple accuracy, such as algorithmic fairness, data privacy (e.g., the EU GDPR's data minimisation principle), and computational efficiency.

A major challenge is that these requirements often emerge after a model has been trained and deployed—whether due to new legislation like the EU AI Act or newly discovered vulnerabilities. Traditional ways to address this post-deployment mismatch fall into two camps, both flawed:

  1. Post-processing methods: Fast, but they often severely degrade the model's utility or accuracy.
  2. Fine-tuning or retraining from scratch: Computationally expensive, time-consuming, and frequently impossible if the original training data is unavailable due to privacy or intellectual property constraints.

The authors ask: Can we efficiently edit models to satisfy new requirements post-hoc, without sacrificing their utility or requiring full retraining?

Methods

The authors frame requirement compliance as a multi-objective optimization problem with two competing terms:

  • Preservation Objective: Minimizes the functional drift between the original and edited model using Jensen-Shannon Divergence (JSD).
  • Requirement Objective: Quantifies compliance (e.g., fairness metrics or sparsity counts).

Instead of solving this optimization problem from scratch for every single model and requirement, the authors use amortized optimization. They train a graph metanetwork (GMN)—a specialized neural network that takes the weights and computational graph of a target network as input and outputs a set of edited weights in a single inference step.

[Original NN Weights] ---> ( Graph Metanetwork ) ---> [Compliant NN Weights]

To make this work, the authors introduce several key technical innovations:

  • Symmetry-Aware Architecture: The metanetwork is designed to be equivariant to parameter symmetries (like neuron permutations), which significantly eases the optimization landscape in weight space.
  • Continuous Relaxations: Many regulatory requirements are inherently discrete and non-differentiable (e.g., counting active features or pruned weights). The authors use softmax-with-temperature relaxations and straight-through estimators to allow end-to-end gradient-based training.

Datasets

To evaluate their approach, the authors constructed two large populations of trained Multilayer Perceptrons (MLPs), consisting of 12,000 models each, based on two widely used UCI repository benchmarks:

  • Adult Dataset: Used to evaluate data minimisation, weight pruning, and bias mitigation (with gender as the sensitive attribute).
  • Bank Marketing Dataset: Used to evaluate data minimisation and weight pruning.

The model populations were generated using diverse architectures, hyperparameters, activation functions, and normalization layers to ensure the metanetwork learned to generalize across different model structures.

Key Insights and Ideas

  • Instantaneous Editing: Once trained, the metanetwork acts as a "regulatory operator," editing target models in milliseconds rather than the hours or days required for retraining.
  • Data-Agnostic Inference: At inference time, the metanetwork requires absolutely no data samples to edit the target model—it operates purely on the model's weight space.
  • Compositionality (Stacking Editors): A highly surprising finding is that independently trained metanetworks can be stacked sequentially (e.g., applying a data minimisation editor followed by a pruning editor). Because the preservation objective prevents catastrophic distribution shifts in weight space, the stacked metanetworks compose cleanly without destroying model utility.

Key Results

The authors evaluated their method against popular post-processing and retraining baselines across three tasks: data minimisation, bias mitigation, and weight pruning. In all cases, the metanetwork approach consistently dominated the Pareto front, providing a better trade-off between utility preservation (low JSD) and requirement compliance.

Data Minimisation

The metanetwork successfully deactivated unnecessary input features while maintaining model functionality better than standard feature selection or knowledge distillation baselines.

Figure 1: Data Minimization on the Adult dataset.

Bias Mitigation

When mitigating gender bias using the Equalised Odds metric, the GMN achieved significantly lower functional divergence than traditional post-processing algorithms like ThresholdOpt or RejectOption.

Figure 2: Bias Mitigation on the Adult UCI dataset. Sensitive attribute: gender

Efficiency

As shown in the paper's evaluation, the metanetwork approach requires only 0.03 seconds to edit a model, matching the speed of naive post-processing while delivering the superior performance typically associated with heavy retraining (which takes upwards of 32 seconds per model).

Limitations and Future Work

  • Task Specificity: Currently, a metanetwork must be trained on a population of models originating from the same underlying task distribution. Developing a truly universal, multi-task model editor remains an open challenge.
  • Architectural Complexity: The proof-of-concept focuses entirely on MLPs. Extending this framework to more complex architectures like Convolutional Neural Networks (CNNs) and Transformers is a critical next step.
  • White-Box Access: The method assumes full access to the target model's internal parameters, meaning it cannot be applied to black-box API-based models.

Summary updated

Abstract

As machine learning models are increasingly deployed in high-stakes settings, e.g. as decision support systems in various societal sectors or in critical infrastructure, designers and auditors are facing the need to ensure that models satisfy a wider variety of requirements (e.g. compliance with regulations, fairness, computational constraints) beyond performance. Although most of them are the subject of ongoing studies, typical approaches face critical challenges: post-processing methods tend to compromise performance, which is often counteracted by fine-tuning or, worse, training from scratch, an often time-consuming or even unavailable strategy. This raises the following question: "Can we efficiently edit models to satisfy requirements, without sacrificing their utility?" In this work, we approach this with a unifying framework, in a data-driven manner, i.e. we learn to edit neural networks (NNs), where the editor is an NN itself - a graph metanetwork - and editing amounts to a single inference step. In particular, the metanetwork is trained on NN populations to minimise an objective consisting of two terms: the requirement to be enforced and the preservation of the NN's utility. We experiment with diverse tasks (the data minimisation principle, bias mitigation and weight pruning) improving the trade-offs between performance, requirement satisfaction and time efficiency compared to popular post-processing or re-training alternatives.
BibTeX
@misc{kalogeropoulos2025metanetworks,
  author = {Ioannis Kalogeropoulos and Giorgos Bouritsas and Yannis Panagakis},
  title = {{Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance}},
  year = {2025},
  eprint = {2512.15469},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.15469},
}
Figure